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ABSTRACT 



This thesis examines the use of random sampling as a 
basis for taking extrapolated deductions in the payment of 
Navy maintenance service contracts. Motivated by increasing 
inspection requirements, NAVFAC has authorized limited use 
of this quality assurance tool. The experience of two 
activities which implemented extrapolated deductions in 
their family housing maintenance contracts is the focus of 
this study. Methods employed are evaluated along with an 
analysis of payment history, contractor performance and 
inspection requirements. The statistical basis used in the 
field tests along with an alternate proposal is analyzed. 
Various issues are identified which should be addressed 
before general guidance is promulgated. Major 
recommendations deal with sample adjustment, computer 
support, inspection methods and the need for oversight. 
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I. 



INTRODUCTION 



A . BACKGROUND 

Random Sampling for Extrapolated Deductions (RSED) 
involves making an inference concerning a population based 
on a simple random sample. The results of that sample are 
then used as a basis for equitable payment across the entire 
contracted service. As a simple example, suppose that a 
sample inspection of services revealed 10% not performed. 
The contracting officer would then estimate that 10% of all 
services were not performed, and reduce the contract price 
by 10%. While payment deductions for observed poor perform- 
ance or nonperformance by service contractors is standard 
practice, the extrapolation of deductions based on random 
sampling to the general population is currently prohibited 
by NAVFAC regulation. This policy has been under review in 
recent years. 

Interest in this technique is a result of the increasing 
role that service contracts play in accomplishing the base 
maintenance function and the subsequent strain placed on the 
inspection force. As part of a move toward formulating Navy 
policy, two activities were granted authority to use 
extrapolated deductions in their Family Housing Maintenance 
Contracts. 
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B. OBJECTIVE 



The objective of this research is to evaluate the field 
use of random sampling for extrapolated deductions in Navy 
maintenance service contracts. This is being done to 
identify issues which should be addressed before general 
authority is granted and guidance promulgated. In addition, 
proposed NAVFAC statistical methodology is evaluated in 
light of information obtained in the course of this study. 

C. SCOPE AND ASSUMPTIONS 

This study focuses on the procedures used to implement 
the field testing of random sampling for extrapolated 
deductions in Navy maintenance service contracts. Air Force 
use of this contracting technique is addressed to provide a 
background and contrast to the NAVFAC effort. Experiences 
of other service contracting organizations with 
extrapolating deductions either within or outside the 
Federal Government was not pursued. It is assumed that the 
reader is familiar with the concept of simple random 
sampling. 

D. RESEARCH METHODOLOGY 

A literature search was conducted to review regulations, 
reports, industrial literature, and policy guidance applica- 
ble to statistical quality assurance and service 
contracting. Contact documentation and history was obtained 
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from the following organizations involved in the initial use 
of extrapolating deductions: 

1. Public Works Center, San Francisco, Ca; and 

2. Public Works Center, Great Lakes, II. 

Additionally, interviews were conducted either in person 

or by telephone at the following organizations: 

1. Public Works Center, San Francisco, Ca; 

2. Public Works Center, Great Lakes, II; 

3. Western Division, Naval Facilities Engineering 
Command, San Bruno, Ca; 

4. Public Works Center, San Diego, Ca; 

5. Public Works Department, Naval Postgraduate School, 
Monterey , Ca ; 

6. Public Works Department, Marine Corps Air Station, El 
Toro , Ca ; 

7. Vandenburg Air Force Base, Lompoc, Ca. 

E. THESIS ORGANIZATION 

This chapter identified the current status of NAVFAC 
regarding the extrapolation of deductions in service con- 
tracting and presents the objectives and methodologies of 
this study. Chapter II puts the study into perspective with 
a discussion of the maintenance service contracting environ- 
ment. Chapter III traces the historical development of 
statistical quality assurance techniques and their applica- 
tion to service contracting. Chapter IV examines random 
sampling, NAVFAC policy concerning RSED, and two statistical 
approaches that have received consideration as a basis for 
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extrapolating deductions in NAVFAC service contracts. 
Chapter V presents findings and conclusions resulting from 
an evaluation of two activities implementation of 
extrapolated deductions in their family housing maintenance 
contracts. Finally, Chapter VI lists recommendations for 
the general use of extrapolating deductions and makes 
suggestions for future research. 
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II . MAINTENANCE SERVICE CONTRACTING ENVIRONMENT 



A. BACKGROUND 

A Maintenance Service Contract (MSC) , commonly referred 
to as simply a service contract, calls for a contractor's 
time and effort rather than a finished product. Service 
contracts are funded by the organization having budgeted 
responsibility for the services acquired. Examples of 
service contracts include refuse collection, guard services, 
grounds maintenance, pest control, and family housing 
maintenance. A MSC is normally entered into for a one year 
period with an option to extend for two additional one year 
periods. However, extensions to the contract cannot exceed 
the fifth anniversary of contract award. [Ref. 1] 

While not a critical distinction for purposes of this 
paper, a MSC is one category of a Facility Support Contract. 
The other category is a Maintenance Construction Contract. 
Due to the nonrepetitive nature of construction and the 
higher degree of importance associated with each unit of 
work, they are not normally amenable to statistical quality 
assurance and will not be a focus of this study. 

Facility Support Contracts are designed to accomplish 
the maintenance and repair of real property facilities. 
They can best be thought of as an extension of the base 
public works force, and in fact replace the in-house forces 
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where a decision has been made to contract out a function 



previously performed by Public Works. 

Service contracts are almost exclusively of the fixed 
price type. It is the belief of NAVFAC that fixed price 
contracts force contractors to perform efficiently. Where 
uncertainty of workload precludes a firm fixed price. 
Indefinite Quantity contracts are awarded on the basis of 
firm unit prices for an estimated workload. This study 
encountered the extrapolation of deductions being applied 
only to firm fixed price contracts, however the technique is 
also applicable to firm unit prices. [Ref. 1] 



B. TREND TOWARDS SERVICE CONTRACTING 

Service contracting in the public sector gained 

attention in 1954 when, in his first budget message to 

Congress, President Eisenhower said: [Ref. 2] 

This budget marks the beginning of a movement to shift to 
. . . private enterprise Federal activities which can be 
more efficiently carried on that way. 

This was followed by Bureau of the Budget Bulletin 55-4 

which implemented the stated policy as follows: [Ref. 3] 

It is the general policy of the administration that the 
Federal Government will not start or carry on any commer- 
cial activity to provide a service or product for its own 
use if such service or product can be produced from 
private enterprise through ordinary business channels. 
Exceptions to this policy shall be made by the head of the 
agency only where it is clearly demonstrated in each case 
that it is not in the public interest to procure such 
products or services from private enterprise. 

Today's policy comes from 0MB Circular A-76, issued in 
1966 and further revised in 1967, 1979 and 1983. It 
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basically states that whenever a commercial activity is 
Federally operated, the cost of operation should be compared 
to the cost of contracting to determine who should do the 
work. A commercial activity (CA) is defined as a product or 
service obtainable from a commercial source. [Ref. 4] 

Although it has been federal policy to contract out 
commercial functions since the 1950s, the program was 
largely ignored until the 1979 revision of A-76 [Ref. 5]. A 
key element of that version required government forces to 
submit a "firm bid" based on a clear Statement of Work 
(SOW) . Contractors would also bid on the requirements 
contained in the SOW. Thus the open market would determine 
the contracting decision. Award would be to the lowest 
responsible bidder if under the government estimate by 10%. 
To ensure that the government estimate was properly 
prepared, it was subject to approval by the Naval Audit 
Service [Ref. 6] . 

Thus, the fact that DODI 4100.33 required a complete 
review of all CA functions and a compliance system was in 
place set the stage for a number of functions formally 
performed by Public Works to be contracted out. 

Data compiled in January 1982 showed that 60% of all 
functions reviewed shifted to contract, at an overall cost 
savings of 19%. Of all NAVFAC functions studied, the areas 
of transportation, grounds maintenance, building mainte- 
nance, and janitorial service accounted for 82% of the total 
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[Ref. 7]. The next few years will undoubtedly see conver- 
sions occur at an ever increasing rate as the great majority 
of reviews are completed. 

C. PERFORMANCE 

Service contracts, while not directly mission related, 
are usually of great concern to the activity commanding 
officer. Visible lack of performance reflects directly on 
his ability to effectively maintain the shore establishment 
entrusted to his care. The result is acute command interest 
in performance, an interest which is reflected throughout 
the contracting organization. 

Acceptable performance in service contracts is often 
difficult to achieve. While a consensus of opinion did not 
emerge during personal interviews, frequently cited reasons 
included a lack of inspector resources, inadequate specifi- 
cations, and a lack of enforcement techniques to motivate 
the contractor. 

D. QUALITY ASSURANCE 

Quality Assurance (QA) is the procedure by which the 
government verifies that it is receiving the services for 
which it contracted and paid for. Quality Assurance 
Evaluators (QAE) are responsible to the Service Contract 
Manager (SCM) for actual monitoring of contractor 
performance. The QA program is contained in the QA Plan, 
it details how the QAE will monitor all aspects of a 
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contractor's performance, although it is not part of the 
actual contract. 

Several surveillance methods may be used as part of the 
QA Plan. Those selected are based on the unique considera- 
tions of an activity. Techniques currently in use are: 

[Ref. 8] 

a) One Hundred Percent Inspection — Appropriate for infre- 
quently performed services or those of great impor- 
tance, it is expensive and time consuming. 

b) Planned Sampling — Designed to inspect some part of the 
contract requirement. It cannot be used to make an 
overall judgment concerning contract performance, but 
is useful where performance in a particular area or 
location is of interest. 

c) Unscheduled Inspection — An impromptu check. 

d) Validated Complaints — Originate with the customer. 
The method is successful to the extent that customers 
are aware of and can distinguish good performance. 
While not recommended as a stand alone procedure, it 
can be used to make payment deductions and require 
rework. 

e) Random Sampling — A statistically based procedure which 
attempts to incorporate the best features of 100% 
inspection and planned sampling. By inspecting a 
sample representative of the entire population, it 
allows a judgment to be made about overall performance 
with a reduced level of inspector resources. A 
properly constructed random sample may also be the 
basis for extrapolating deductions. However, the 
extrapolation of deductions is currently not allowed 
by NAVFAC regulation and is the focus of this study. 



E . ENFORCING PERFORMANCE 

If the government accepts anything less than 100% of 
contract bid requirements, the contracting officer must take 
some appropriate action. Not to do so could indicate that 



16 



the government has overstated its requirements resulting in 
an illegal contract. 

Other than the routine identification of contractor 
deficiencies, negative motivators form the basis of NAVFAC 
approved techniques to enforce performance. Contract 
actions other than payment reduction are all designed to 
threaten and support eventual default for failure to 
perform. The effectiveness of these techniques are 
questionable because of the difficulty in sustaining 
termination of a borderline contractor. In addition, the 
prospect of extended litigation and interruption in services 
must be considered. Conversations with contracting officers 
indicate that "living" with less than acceptable performance 
until contract expiration is the best of a bad situation. 
It is reasonable to assume that a savvy contractor is aware 
of the contracting officer's dilemma. 

The most effective performance motivator currently 
permitted is the withholding of payment for unsatisfactorily 
performed services. Withholding payment assumes a strong 
profit motive on the part of a capable contractor, a "fair" 
contract under which reasonable profits are possible, and a 
schedule of deductions which results in reduced profits when 
exercised. Other possible contractor motivations such as 
growth or maintenance of the workforce do not appear valid 
in service contracting. Contractors generally do not staff 
for a job until contract award, and a ready supply of 
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unskilled or semiskilled laborers makes adjustment of the 
workforce an easy thing to do. 

Current NAVFAC regulations allow deductions for work not 
acceptably performed, unless the contractor is given the 
opportunity to re-perform the work and does so satisfactor- 
ily. If feasible, rework is preferable to deductions 
because of the emphasis on performance. When taken, deduc- 
tions are in accordance with individual bid items if an 
Indefinite Quantity contract, or an agreed-upon schedule of 
deductions for Firm Fixed Price contracts. However, this 
contract action is limited by the fact that appeals boards 
and courts uniformly hold "penalty” provisions unenforce- 
able. Section 9-302.1 of the Navy Contracting Manual 
applies: [Ref. 1] 

No contract may provide for penalties for non-performance, 
as such are legally unenforceable. However, a contract 
may provide for liquidated damages, which is an advance 
agreement between parties to a contract as the damages one 
party will suffer if the other fails to perform . . . Note 
that if a contractor does not perform work, it would be 
improper to withhold payment for that work AND also charge 
him the full cost of having the work done by others, for 
that would result in the Navy having work done at the 
expense of others. 

Liquidated damages of 10% are assessed whether or not 
the work is subsequently performed by the contractor to 
compensate the government for increased administrative 
costs. Where government forces re-perform the work, a 20% 
liquidated damage charge is levied along with the full cost 
of the government effort. [Ref. 1] 
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Current NAVFAC regulations allow deductions for observed 
defects only. The extrapolation of deductions based on 
random sampling is prohibited [Ref. 8]. This policy 
resulted from the lack of an acceptable statistical tech- 
nique and questions as to the legality of extrapolated 
deductions. However, NAVFAC has recognized the potential of 
extrapolating deductions to further motivate the contractor, 
or at least to better match payment for services received. 
Therefore they have initiated efforts in recent years to 
define a methodology under which extrapolation might be 
authorized for general use. In support of this effort, 
NAVFAC has authorized trial tests of extrapolating deduc- 
tions in family housing maintenance contracts at PWC San 
Francisco and PWC Great Lakes. 
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III. HISTORICAL OVERVIEW OF STATISTICAL 
QUALITY CONTROL 



A. EARLY EVOLUTION 

Since man began to manufacture and produce products, 
there has been concern over quality of output. As far back 
as the Middle Ages, the medieval guilds insisted on 
apprenticeships and required that those seeking to become 
master craftsmen offer evidence of their ability. Such 
rules were in part a form of quality control. In more 
modern times, factory inspection and research, activities of 
professional societies, and various regulations have sought 
for years to assure a quality of output. Quality control 
has thus had a long history. 

On the other hand, statistical quality control is rela- 
tively new. The science of statistics itself goes back only 
about 300 years. The theory of probability originated in 
1654 when Pascal, a French philosopher and mathematician, 
teamed up with Pierre Fermat to develop the new science. 
Early applications were made in astronomy and physics, but 
it was not until the 1920 's that statistical theory began to 
be applied effectively to quality control. It should be 
remembered though, that even before the 1920 's, industry was 
learning to do things more scientifically. Techniques such 
as the GANTT chart, under principles developed by Taylor, 
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Gilbreth and others were a part of the movement called 
"scientific management." 

Walter Shewhart, of the Bell Telephone Laboratories, was 
among the first to apply statistical methods to the problem 
of quality control [Ref. 1]. Two other Bell System men, 
H.F. Dodge and H.G. Romig, took the leadership in developing 
the application of statistical theory to sampling 
inspection. The culmination of their work being the now 
well-known Dodge-Romig Sampling Inspection Tables. An 
immense amount of work went into the preparation of these 
tables, for at that time hand calculators were the only 
available aid to speedy calculations. These tables were 
published in 1941 [Ref. 2]. The work of Shewart, Dodge, and 
Romig constitutes much of what today comprises the theory of 
statistical quality control. 

When this new inspection plan was proposed there were 
many skeptics. Therefore, in the initial trials, after 
performing sampling inspection, every lot was then inspected 
100% to check conformity. The checks showed that sampling 
did give good results and with much less time and money used 
for inspection. After its initial application in the 
central offices of Western Electric Company, the new proce- 
dures were tried in the Hawthorne manufacturing plant in 
Chicago, and proved a big success. 

The rate of adoption of these newer methods were slow in 
the United States. Professor Freeman, who was promoting 
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statistical quality control at Massachusetts Institute of 
Technology, ascribed this sluggish response of the early 
years to: (1) the conviction of American production 
engineers that their principal function was to improve 
quality control through advances in technology, (2) a 
prevailing attitude that the laws of chance had no place in 
a scientific field, and (3) the scarcity of adequately 
trained industrial statisticians. In 1937, probably not 
more than a dozen single enterprises in American mass- 
production industry had introduced the new techniques into 
their ordinary operations. [Ref. 3] 

B. INFLUENCE OF WORLD WAR II 

World War II overcame the initial coolness of American 
industry toward statistical quality control. The armed 
services began to enter the market as large consumers of 
American output and, as such, had an increasing influence on 
quality standards. A group of engineers from Bell Telephone 
were brought to Washington to develop a sampling inspection 
program for Army Ordnance. This step was made necessary by 
the bottlenecks occurring at inspection stations during full 
scale production. 

The group from Bell faced a sampling problem of unprece- 
dented size. It was necessary that all vendors be treated 
fairly; it was necessary to get enormous quantities of ord- 
nance material quickly; and it was necessary that arms and 
ammunition be reliable and safe. The "acceptable quality 
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control" system that was developed encompassed the concept 
of protecting the consumer from getting unacceptably defec- 
tive material, and encouraging the producer in the use of 
process quality control. This was done by varying the 
quantity and severity of acceptance inspections in direct 
relation to the importance of the characteristics inspected, 
and in inverse relation to the cjuality level indicated by 
those inspections [REf. 4]. The same men also carried out 
an extensive training program to acquaint government person- 
nel with the new procedures and tables. Many terms used 
extensively today, most notably "producers risk" and 
"consumers risk," had their origins in this original effort. 

Wartime research in statistical quality control was 
carried out primarily by the Statistical Research Group, 
Columbia University, Applied Mathematics Panel, Office of 
Scientific Research and Development. This research group 
was organized in July 1942, and continued in existence until 
September 1945. Members of the staff were drawn from 
universities around the country. It advised and assisted 
the Army and Navy on statistical aspects of problems arising 
in their activities. Included in its contributions to sta- 
tistical quality control was the preparation of the Navy 
manual on sampling inspection by attributes [REf. 5]. 
Inspection by attributes requires that the inspected item be 
categorized either as totally acceptable or not acceptable. 
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Professor A. Wald, also of the Statistical Research 
Group, developed the idea of sequential sampling. In 
sequential sampling, the sample size is not determined in 
advance. The sequential procedure is to inspect one item at 
a time. The inspection is continued until enough cumulative 
evidence has been gathered to either accept or reject a lot. 
A pre-assigned risk of making incorrect decisions controls 
the ultimate same sample size. This procedure allows very 
good lots to be accepted quickly and bad lots to be rejected 
quickly. It is only with lots of doubtful quality that 
extensive sampling is required. This analysis was deemed so 
important by the U.S. Government that it withheld publica- 
tion of Wald's original paper until June 1945. [Ref. 5] 

C. MILITARY STANDARDS (MIL-STD-105 ) 

1 . Development 

In the post war period there were two systems of 
sampling plans and tables. Army tables and Navy tables. The 
procedures and tables were alike in general plan and struc- 
ture but differed in a number of respects. In 1949, a basic 
Military Standard, MIL-STD-105, was developed using material 
from both service tables. Military Standard 105D, the 
current version, is the outcome of a study by an American- 
British-Canadian working group that sought to derive a 
common standard for the three countries. MIL-STD-105D was 
issued by the U.S. Government in 1963. In 1971, it was 
adopted by the American National Standards Institute as ANSI 
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Z1.4 and in 1973, it was adopted by the International 
Organization for Standardization as ISO/DIS-2859 . [Ref. 6] 

2 . Acceptance Sampling 

Acceptance sampling is the method of evaluating a 
group of units, drawn from a production lot, in order to 
determine the acceptability of the whole lot. It has been 
widely accepted as the preferred method of acceptance con- 
trol. MIL-STD-105D provides tables which facilitate the 
implementation of an Acceptance Sampling by Attributes plan. 
The focus of the plan is an acceptable quality level (AQL) . 
An AQL represents that quality level which, for purposes of 
sampling inspection, can be considered satisfactory as a 
process average. [Ref. 15] 

In addition to an initial decision on an AQL, it is 
also necessary in applying MIL-STD-105D to decide on an 
"inspection level." This determines the relationship 
between the lot size and the required sample size. Three 
inspection levels are provided for general use with guidance 
for switching between them. 

For a particular sample size and AQL, the tables 
will specify a reject number. When the number of defects in 
a sample is equal to or greater than the reject number, the 
entire lot is judged to be unacceptable. The sample size, 
reject number, and AQL define an "operating characteristic" 
curve for a particular plan. This curve depicts the 
relationship between the actual defect rate submitted for 
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inspection and the corresponding probability of acceptance 
of such lots by the sampling plan. It is based on the 
binomial distribution. 

The nature of acceptance sampling is explained by 
Siegmund Halpern: [Ref. 15] 

Inasmuch as only a portion of the whole lot is inspected, 
there is always a risk that the quality of the sample will 
not reflect the quality of the lot. As a result, produc- 
tion lots may, at times, undeservably be rejected and 
faulty lots accepted. The former will be of interest to 
the producer. He wants to minimize the risk of having 
good lots rejected. This risk is identified as the 
producer’s risk. The risk of accepting bad lots in a 
sampling scheme is, of course, the primary concern of the 
consumer. This risk is called consumer's risk. The 
method by which both the producer's risk and the 
consumer's risk are quantified statistically is by 
constructing an operating characteristic curve. 

Acheson Duncan also comments; [Ref. 9] 

It is to be emphasized that the purpose of acceptance 
sampling is to determine a course of action, not to 
estimate lot quality. Acceptance sampling specifies 
procedures that, if applied to a series of lots, will give 
a specified risk of accepting lots of a given quality. 

Appendix A illustrates how the sampling plans 
contained in MIL-STD-105D are used. A typical operating 
characteristic curve is also presented. 

3 . Application to Air Force Service Contracting 

Random sampling as a quality assurance tool for 
monitoring contractor services was introduced to the 
military by the Air Force in 1979 with AFR 400-28. MIL-STD- 
105D provided the basis for the random sampling procedures 
contained in the regulation. [Ref. 16] 
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Originally designed to determine the acceptance or 
rejection of manufactured lots, MIL-STD-105D was easily 
adapted to the service contracting environment. Specific 
instances of work could be judged "defective" or acceptable, 
as required in attribute sampling, the same as a manufac- 
tured part. Lot size was normally determined by the number 
of services provided in a month, which corresponded to a pay 
period. A major difference between the use of acceptance 
sampling for manufactured items and evaluating service con- 
tract performance is that a "lot" of services could not be 
totally rejected if found unacceptable, since it had already 
occurred. Therefore, contract actions based on the results 
of acceptance sampling must recognize that some satisfactory 
work has taken place for which the contractor is due 
payment. 

AFR 400-28 contains tables derived from MIL-STD-105D 
that show acceptance and rejection numbers corresponding to 
specified acceptable quality levels and sample sizes. The 
acceptance number is the maximum number of defects allowed 
before a sample is considered unsatisfactory. When the 
number of defects in the sample exceeds the acceptance 
number, contractor performance is deemed unsatisfactory and 
contract payments can be reduced for the service provided 
during the sample period. 

Allowable AQLs are limited to those which appear in 
MIL-STD-105D. AFR 400-28 calls for the establishment of 
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reasonable AQLs as part of the original CA solicitation. 
Thus contractors bid on performance requirements of less 
than 100% for various services. Contractors are also 
provided with the activities QA plan as an enclosure to CA 
solicitations. This is done to inform prospective bidders 
as to how the contract will be monitored and to help them 
set up their own quality control program. It is not, 
however, a part of the contract. 

The extrapolation of deductions is a key feature of 
AFR 400-28. When the random sampling method of surveillance 
is used and the service is rejected, contractor payments may 
be reduced in the same proportion for the entire lot as the 
percentage of defects found in the sample. For example, if 
a rejected sample disclosed that 15% of an inspected service 
was defective, contract payments for the service may be 
reduced by 15%. However if the number of defects found do 
not equal or exceed the reject number, the services are 
judged satisfactory and no deductions are taken. 

Numerous problems have been experienced at the 
activity level in implementing AFR 400-28. An audit con- 
ducted by the Air Force Audit Agency in 1985 to evaluate 
quality assurance of base-level service contracts reported 
the following findings: [Ref. 17] 

a) for seven of twenty contracts reviewed, QAEs made 
errors in the use of random number tables; 

b) for fifteen of twenty contracts reviewed. QAEs did 
not correctly use random sampling techniques due to 
the complexity of AFR 400-28; 
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c) for ten of twenty contracts reviewed. QAEs did not 
properly apply accept/ reject numbers due to the com- 
plexity of the AFR 400-28 acceptance tables; and 

d) sampling plans were not an effective means of deter- 
mining contractor's performance as the plans used fre- 
quently required the acceptance of services even 
though the percentage of defects occurring in the 
sample was approximately twice the defect rate per- 
mitted in the contract. 

The Air Force is now investigating a simplified 
random sampling process based on Indifference Quality 
Levels. In such a plan, the government and the contractor 
equally share in the probability of acceptance or rejection 
at a specified quality level [Ref. 17]. 

4 . Navy Rejection of MIL-STD-105D 

NAVFAC service contracts are written requiring 100% 
performance. Thus the Navy is more interested in a statis- 
tical methodology which will provide information as to 
actual contract performance rather than the accept/reject 
hypothesis testing of MIL-STD-105D. 

A random sampling procedure was designed by NAVFAC 
in 1979 to determine the level of contractor performance. 
Published in the MO-327 [Ref. 8], which is the Navy equiva- 
lent of AFR 400-28, it was not intended to be a statistical 
basis for extrapolating deductions. The sampling plan 
contained in MO-327 made use of an AQL that was not releasa- 
ble to the contractor. This AQL differed from the one used 
in MIL-STD-105D in that it was defined to be "tolerable" 
performance rather than acceptable. 
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The new contract surveillance technique was not 
widely used. Conversations with contract administrators 
indicated that this method of random sampling was hard to 
understand, difficult to administer, and provided question- 
able results upon which to base contract actions. Adding to 
that the fact that extrapolated deductions were not 
permitted, there remained little incentive for its use over 
the more understandable planned-sampling technique. 

In 1982, NAVFAC determined that the extrapolation of 
deductions was appropriate as long as there was a degree of 
confidence that deductions made were reasonable and not 
construed as penalties. Their approach is studied in the 
following chapter. 
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IV. RANDOM SAMPLING FOR EXTRAPOLATED DEDUCTIONS 



A. INTRODUCTION 

Random Sampling for Extrapolated Deductions (RSED) 
involves making an inference concerning a population based 
on a simple random sample. The results of that sample are 
then used as a basis for equitable payment across the entire 
contracted service. 

There are three reasons why the concept of extrapolating 
deductions is supported by NAVFAC at this time: (1) it is 
believed to provide extra motivation for the contractor 
through the potential of increased deductions, (2) extrapo- 
lation results in a better match of payment to services 
received, and (3) the ability to extrapolate deductions 
makes the use of random sampling more attractive as a 
quality assurance tool. Given the current service 
contracting environment, the ability to obtain acceptable 
evaluation of contractor performance with reduced levels of 
inspector resources would appear to be the predominant 
issue . 

The application of extrapolated deductions must be based 
on a statistically valid random sample if a protest is to be 
upheld. Most activities have had no experience with random 
sampling and the guidance available is limited in value. 
The next section looks at the considerations involved in 
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employing random sampling in a service contracting 
environment . 

B. RANDOM SAMPLING 

1. Characteristics 

A properly constructed random sample must ensure 
that every possible sample of a given size has an equal 
chance of being selected. Additionally, the process being 
inspected must be homogeneous and produce defects in a 
random manner. If this is not the case, a selection bias 
may result which causes the attributes of the sample to be 
different from the population. [Ref. 18] 

This theoretical ideal, approachable in manufactured 
lots, cannot realistically be attained in service 
contracting. Some rooms are going to be harder to clean 
than others. A bus service will be more likely to lag 
behind schedule during certain hours of the day. If grounds 
maintenance is provided at two different bases, a SCM would 
not be happy with a sample which required 95% of the 
inspections at one base. Thus, a sampling plan must be 
devised which will "reasonably" represent the population of 
services. A general rule would be to adopt a selection 
method that will give every member of the population an 
equal chance of being selected. In general, it is best to 
use random numbers. These will hold up in court [Ref. 9]. 
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2 . Advantages 



The use of random sampling as a quality assurance 
tool has inherent advantages over other inspection tech- 
niques. The most obvious is that it provides a cost effec- 
tive way of estimating the true level of contractor 
performance. The fact that it may now be used as a basis 
for extrapolating deductions makes it all the more attrac- 
tive. Some other benefits of random sampling offered by 
contract administrators are: 

a) Requires less supervision of inspectors to make sure 
they are using their time effectively; 

b) is not susceptible to the predictability which may be 
involved in planned sampling; 

c) allows inspection of services to be transferred easily 
from one inspector to another as no previous knowledge 
of the contract is required. 

NAVFAC has recommended that random sampling be 
applied to service contracts possessing the following 
characteristics: [Ref. 8] 

a) Composed of a large homogeneous population of work 
units , 

b) individual work items not of a critical nature, 

c) population size estimable, 

d) inspector travel time not excessive. 

3 . Problems in Practical Use 

Although there would seem to be numerous advantages 
involved with the use of random sampling, even without 
extrapolated deductions, its use seems to be very limited 
within the NAVFAC community. Conversations with various 
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contract administrators indicate that a lack of familiarity 
with random sampling procedures, the complexity of writing 
amenable specifications, and the confusing guidance in the 
MO-327 are at the heart of their reluctance. 

Probably the major impediment to instituting random 
sampling plans, with or without extrapolated deductions, is 
the lack of random sampling guide specifications for 
individual contract types. Contract administrators are 
either not willing or able to get bogged down in the details 
of generating performance specifications and quality 
assurance plans which properly incorporate random sampling. 
For that reason, a great many services amenable to random 
sampling are being inspected in a judgmental manner with 
poor performance defined as an intolerable level of customer 
complaints . 

Janitorial services would seem to be a prime candi- 
date for random sampling, yet interviews with contract 
administrators revealed confusion and frustration where 
attempts to write a suitable plan were made. This confusion 
is easily understood. Questions exist over what the sampled 
unit should be. Should it be individual rooms, groups of 
rooms, buildings, or the specific acts of emptying ashtrays 
and cleaning windows? Also, how is a representative sample 
taken when conditions vary? Suppose that a BOQ building is 
randomly selected for inspection. Can the percent of rooms 
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found to be nonconforming be used as an indicator of overall 
performance across the various buildings on base? 

The design of random sampling plans is beyond the 
scope of this paper. It is addressed because of its 
discovery as an issue in the course of the research. Due to 
the complexities involved with designing random sampling 
plans and the lack of specific guidance, I would not be 
surprised if the majority of random sampling plans which in 
effect are of poor statistical validity. Contractors have 
had no reason to challenge these procedures either because 
they did not understand them, or because they were not being 
used as a basis for deductions other than observed deficien- 
cies. It should be anticipated that the extrapolation of 
deductions will cause many of these sampling plans to come 
under intense scrutiny. 

Another impediment to the use of random sampling has 
been the difficulty of generating random numbers and apply- 
ing it to the population. Appendix B contains instructions 
from the MO-327, which provides the guidance for random 
sampling and tables for random number generation. Sample 
generation by the manual method can be seen to be time 
consuming and difficult to apply objectively. The tendency 
would be for users to become frustrated with unusable 
numbers and just resort to making up numbers on their own. 
While NAVFAC recognizes the shortcomings of the random 
number table and now advocates the use of computer programs 
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for generating samples, they still recommend the use of the 
tables where a computer and program are not available. 
[Ref. 19] I believe that a computer based system is a 
necessary part of establishing a viable program with 
complete command support. 

C. EXTRAPOLATING DEDUCTIONS 
1. Background 

NAVFAC did not adapt the statistical methods of MIL- 
STED-105D for surveillance of NAVFAC service contracts 
because it called for acceptance of work at levels below 
100%. Additionally, it allowed large deviations in the 
observed sample above the AQL before it was rejected. 
Reservations about the legality of extrapolated deductions 
was also expressed. [Ref. 20] 

In 1983, NAVFAC adopted the thinking that extrapo- 
lated deductions would be an effective contractor motivator. 
It was also thought that it would provide an additional 
incentive for activities faced with scarce inspector 
resources and increasing levels of contracted services to 
adopt random sampling. The technique would be implemented 
when a methodology was available to take deductions fairly, 
with a high degree of confidence that they would not take 
the form of punishment. Dr. Douglas Montgomery of the 
Georgia Institute of Technology was hired as a consultant to 
develop a statistical basis and procedure for extracting 
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these deductions [Ref. 20] . Working with NAVFAC, a 
methodology was ready for field use in 1984. 

The ultimate form that extrapolated deductions will 
take is still in doubt. NAVFAC is currently working on a 
new statistical basis which is more in line with its 
contracting philosophy. The elements of that methodology 
will also be looked at in this chapter. 

2 . Legal Considerations 

The legal basis for extrapolating deductions is 

derived from the Federal Acquisition Regulations: [Ref. 21] 

If any of the services do not conform with contract 
requirements, the Government may require the contractor to 
perform the services again. . . . When the defects in 

services cannot be corrected by reperformance, the Govern- 
ment may . . . (2) reduce the contract price to reflect 

the reduced value of the services performed. 

A key concept is that the reduction in price must 
correspond to the cost of the services. A case that applied 
this ruling was Environmental Aseptic Services Administra- 
tion and Larson Building Care Inc . [Ref. 22]. This case 
involved an Air Force janitorial contract. The protesters 
complaint was that sampled services, which in this case was 
"cleaned rooms," subsumed several required tasks (e.g., 
aseptic floor, furniture, fixtures, drapes, and trash) . The 
Performance Requirements Summary provided that if a task 
fails, the room failed for the day. In other words, if the 
contractor failed to clean the ashtrays, he would suffer the 
same deduction as though he had failed to perform all 
required tasks in the room. Thus the extrapolated deduction 
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greatly exceeded the value of the one task not performed. 
The GAO ruled in the case that the deduction was tantamount 
to a penalty, and its inclusion in the Invitation for Bids 
unnecessarily raises the governments cost and has an adverse 
effect on competition. 

Closely tied to the issue of reasonable cost deduc- 
tions is the treatment of liquidated damages. The Federal 
Acquisition Regulations state the following policy: [Ref. 
23] 

Liquidated damages clauses should be used when both (1) 
the time of delivery or performance is such an important 
factor . . . that the Government may reasonably expect to 
suffer damage if the delivery or performance is delinquent 
and (2) the extent or amount of such damage would be 
difficult or impossible to ascertain or prove. . . . The 
rate of liquidated damages used must be reasonable and 
considered on a case-by-case basis. . . . Liquidated 
damages fixed without any reference to probable actual 
damages may be held to be a penalty and therefore 
unenforceable . 

The Navy Contracting Manual, P-68, allows liquidated 
damages to be assessed o FSCs at the rate of 10% of the 
price of the service whether or not it is subsequently re- 
performed. The extrapolation of deductions has caused some 
confusion in this area. Do liquidated damages apply to the 
estimated portion of unperformed work? 

Original NAVFAC guidance indicated that the extrapo- 
lation of liquidated damages was proper [Ref. 24]. Later 
guidance called for liquidated damages to be applied only to 
items re-performed, which would prohibit their extrapolation 
[Ref. 19]. More recent discussions with NAVFAC indicate 
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that the policy will be to take liquidated damages for 
observed deficiencies, whether or not it is subsequently re- 
performed. 

The key question to be asked concerns the reason for 
liquidated damages. If it is to compensate the government 
for increased administrative costs associated with re- 
inspection, then it is appropriate to apply liquidated 
damages only in instances where the item is re-performed. 
However, if the basis of the liquidated damages is to 
compensate the government for the interruption of normal 
activities associated with dirty chalkboards and unemptied 
dumpsters, then re-performance is not a consideration and 
the damages remain. Also, the damages associated with the 
estimated portion of nonperformance would be just as real as 
the inspector observed deficiencies. The fact that NAVFAC 
now advocates that liquidated damages be applied for 
observed deficiencies would indicate the latter as a basis. 
Therefore, the extrapolation of liquidated damages would 
appear defensible in court. The greater amount of deduc- 
tions assessed would also act as an enhanced motivating 
force. 

Another important issue from a legal standpoint is 
the implications of not obtaining a sample by procedure. 
For example, suppose an inspector fails to obtain the sample 
required by procedure. Does that invalidate the sample for 
extrapolation purposes? From the standpoint of violating 
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the procedure itself, apparently not. The GAO has ruled 
that when regulations set out instructions clearly for the 
benefit of the government, the agency's failure to comply 
with it does not provide a basis for protest. [Ref. 25] 

As far as the sample size itself, the government's 
failure to select an adequate sample has been the basis for 
upholding protests [Ref. 25]. However, that case involved a 
sampling scheme which was part of the contract. NAVFAC has 
no plans to make their quality assurance program a contract 
requirement. Even in the case where a sampling scheme has 
been included as a part of the specifications, the Armed 
Service Board of Contract Appeals has ruled that a deviation 
in sampling procedure is allowable where "no appreciable 
difference in results occurs" [Ref. 26]. I agree with the 
NAVFAC policy not to include quality assurance procedures in 
the contract. The government must reserve the right to 
remain flexible in this area. At the same time, contrac- 
tually specified quality assurance would open up contract 
administrators to numerous protests. 

D. STATISTICAL BASIS FOR FIELD TESTING 
1. Introduction 

The statistical method devised by D.r Montgomery for 
use in the NAVFAC field tests are easily incorporated into 
tables amenable for field use. These tables require no 
knowledge of statistics for application. However, a lack of 
appreciation for the relationships which form the basis of 
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the tables means that the potential for misusing them 
exists. Lacking specific guidance, activities may unneces- 
sarily increase their administrative burden in cases of 
mismatched samples and populations. It should be emphasized 
that the motivation behind extrapolated deductions is to 
better match pay to services using minimal inspector 
resources. This section looks at the statistical basis of 
the field tests and discusses the various components upon 
which the tables are based. 

2 . Use of the Tables 



Tables approved by NAVFAC for use in monitoring 
service contracts and taking extrapolated deductions were 
distributed to Engineering Field Divisions and Public Works 
Centers in 1984. Guidance in use of the tables (see sample 
tables in Appendix C) was provided by NAVFAC as follows: 
[Ref. 24] 

In order to use the tables, the user must first establish 
the "Critical Performance Level" (CPL) which is defined as 
the level of performance by the contractor that the 
government considers to be adequate and is expressed as a 
percentage. The CPL is established during the specifica- 
tion writing phase and should be entered into the Perform- 
ance Requirements Summary. For a given CPL and population 
size, the tables give the user a sample size and a 
Critical Number of Defects (CND) . The CND is the number 
of defects in the sample that would signal poor contractor 
performance and is the threshold for considering taking a 
deduction based on a statistically estimated number of 
defects. When the number of defects observed in the 
sample equals or exceeds the CND, the number of defects in 
the population is estimated by multiplying the population 
by the percentage of defects found in the sample. When 
the number of defects observed in the sample is less than 
the CND, deductions will be taken for observed defects 
only. . . . The Normal Surveillance Table will usually be 
used for a new contract. The Tight Surveillance Table 
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should be used if observed defects exceed two-thirds of 
the CND for two consecutive months. The Reduced Surveil- 
lance Table should be used if the observed defects are 
less than one-third of the CND for two consecutive months. 
Normally when Reduced Surveillance Tables are used, deduc- 
tions will be taken for observed defects only. 

Use of the term CPL (equal to 100-AQL) was NAVFAC's 
way of stating that 100% performance was being contracted 
for, but that a somewhat lower level would be tolerated 
before adverse contract action would be initiated. The 
inclusion of the CPL in the contract was NAVFACs first 
concession in writing that less than 100% performance was 
acceptable. This change in policy was made necessary by the 
fact that the CPL was the key factor in taking extrapolated 
deductions, and in the course of explaining deductions to 
the contractor it would have become known anyway. 

Opinion is mixed whether or not the CPL being part 
of the bid package would lead to lower bids by contractors 
planning to under-perform. The incentive to do so would 
seem to be there. For example, if the CPI was 90%, a 
contractor could cut his bid by 10% and plan to only perform 
90% of the work. If he could maintain his actual perform- 
ance at 90%, he could avoid extrapolation 95% of the time. 
Although he would have to rework observed deficiencies or 
suffer the specific deductions, these would only be a small 
percentage of the actual deficiencies which existed. The 
result of all this would be a borderline contractor with 
high administrative costs. This possibility points out a 
weakness of including the CPL in the contract under this 
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type of system. Limiting the CPL to the QA plan would allow 
it to be changed at a later date if it became necessary. 

It should be noted that the method approved is very 
similar to that used by the Air Force. For a given perform- 
ance requirement and population size, a CND (reject number) 
is established at which time extrapolation can be applied. 
The major procedural difference is that below the CND the 
Navy would still deduct for observed defects, while the Air 
Force did not. This difference was due to the Navy's basic 
requirement being 100% performance. 

3 . Mathematical Basis of Field Method 

The mathematical basis of the tables used for the 
field test is the normal approximation of the hypergeometric 
probability distribution. The sample sizes shown in the 
tables were calculated using the relationship of equation 
( 1 ) : 
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where: 

n = required sample size 
CPL = critical performance level 
N = estimated population size 
Z = normal one-sided z-statistic 

W = width of the confidence interval = ( 1-CPL) /2 
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The z-statistic is dependent on the confidence level 
desired. For normal inspection the confidence level is 95% 
and the corresponding z-statistic is 1.645. This means that 
the resulting sample will ensure that the observed defect 
rate is within the specified confidence interval 95% of the 
time. Tightened inspection equals 99% confidence, while 
reduced equals 90%. 

4 . Effect of CPL 

The confidence interval (W) can be seen to be a 
function of the CPL. It is perhaps easier to think of it as 
one-half the allowable defect rate. Thus the greater the 
CPL, the more narrowly defined is the confidence interval, 
and the greater is the sample size required. This relation- 
ship may be undesirable due to the lack of specific guidance 
in setting CPLs. Contract administrators have interpreted 
the CPL to be a level of acceptable performance from an 
average contractor. Thus easily performed services should 
rate a high CPL regardless of the importance of the service. 
Because the confidence interval then becomes such a small 
number, which is subsequently squared, it has an overriding 
effect on the sample size. From the normal table (see 
Appendix C) it can be seen that if the population estimate 
is 1000 units of work, a shift in the CPL from 90% to 95% 
would result in the required sample size being doubled. 

If the confidence interval is going to vary 
inversely with the CPL, a distinction should be made between 
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"critical" services and those which happen to rate a high 
CPL due to the ease with which they can be performed. 
Perhaps the latter category could be inspected at a reduced 
level of confidence. This would be consistent with the 
efficient use of inspector resources. 

The use of the CPL term in equation (1) is only a 
best estimate of actual performance, which is what is 
actually mathematically required to make the equation valid. 
If the actual performance is less, the actual confidence 
level achieved will be less. NAVFACs rationale for 
accepting lower confidence levels when contractor 
performance is below the CPL is that greater attention will 
be focused on the contractor's performance by various other 
methods. A plot of this relationship for a 95% confidence 
level (normal surveillance) and CPLs of 90% and 97% is shown 
in Figure 4.1, a population of 300 was assumed. It can be 
seen that achieved confidence at realistic performance 
levels still gives the benefit of the doubt to the contrac- 
tor. Therefore, whether or not other actions are antici- 
pated, very poor performance does not invalidate 
extrapolation . 

5 . Effect of Sample Size 

In Section C-2 of this chapter, the implications of 
not taking the appropriate sample size was discussed. 
Figure 4.2 shows a plot of the relationship between sample 
size and confidence level for populations of 100 and 500, 
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normal sampling and a CPL of 90% are assumed. Where actual 
samples fall below the required sample, achieved confidence 
gradually decreases. This is especially true where large 
samples are involved. For example, if the sample taken for 
a population of 500 was only half the required 82, achieved 
confidence would still be in excess of 86%. It can be 
concluded that failure to reach a designated sample is 
probably not going to appreciably affect the results. 
Therefore guidance on this matter should be promulgated to 
avoid the situation of activities invalidating their samples 
when there is no need to do so. 

6 . Effect of Population Estimate 

Similar to the situation of not obtaining the proper 
sample size is the almost monthly problem of not properly 
estimating the population size. Analysis of the relation- 
ship between confidence level and population size shows that 
this situation is of almost no consequence. Take the case 
where the estimated population is 500 units and the desired 
confidence level is 95% (CPL = 90%, sample size = 82). If 
the actual population turns out to be 400, the resulting 
confidence level will be 95.5%. At an actual population of 
600, the original sample of 82 will still give a confidence 
level of 94.7%! In fact, whenever the original sample size 
is greater than about 30, the achieved confidence level 
should be sufficient to support extrapolated deductions 
using the original CND. Again, there is no guidance on this 
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matter and activities may try to unnecessarily match sample 
sizes to ultimate populations at the expense of manpower and 
possibly statistical validity. It would appear to be the 
case that in all situations where random sampling is appro- 
priate, the original sample should be sufficient to support 
extrapolated deductions or any other contract action. 

7 . Operating Characteristics 

The CND is calculated by determining the defect rate 
which exceeds the CPL plus the confidence interval at the 
specified confidence level. Equation (2) illustrates this 
relationship: 



CND = n[CPL + z y ^ 2 ) 

The application of equation (2) is equivalent to a 
standard hypothesis test. The statement being made at a 
normal level of inspection is that if the CND is reached, we 
are 95% confident that the actual defect rate is equal to or 
greater than the allowable defect rate. 

Figure 4 . 3 contains an operating characteristic 
curve for various CPLs. It can be seen that while this 
methodology is biased toward the contractor, it does exhibit 
higher levels of producer risk than MIL-STD-105D. A good 
point of comparison is an actual performance rate of 85% and 
a CPL of 90% (AQL = 10%) . The NAVFAC statistical basis will 
accept this level of performance just over 50% of the time. 
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ACTUAL PERFORMAHCE PERCEHT 

Figure 4.3 Characteristics of Normal Sampling, sample=50 
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while the Air Force would accept it 85% of the time using 
MIL-STD-105D (see Appendix A for MIL-STD-105D) . 

E. PROPOSED STATISTICAL BASIS FOR FUTURE USE 

A problem with the statistical basis used in the field 
tests, as viewed by NAVFAC, is that it did not allow for 
extrapolation until contractor performance was below the 
CPL. This implied that performance above the CPL was 
acceptable. Yet NAVFAC is adamant that service contracts be 
written for 100% performance. Thus, a new method for taking 
extrapolated deductions is being developed by NAVFAC. It is 
planned that when ready for field use, this method will 
replace the method used in the field tests. This section 
describes the characteristics of the new method as it 
currently exists. 

The proposed statistical basis for taking extrapolated 
deductions in the future is more in line with NAVFAC ’s 
policy of contracting for 100% performance and taking 
deductions where evidence of nonperformance exists. Code 
named RSED V3.0, this methodology seeks to determine the 
contractor's actual level of performance. Deductions are 
then extrapolated without regard to a CPL. The concept of a 
CPL is utilized to the extent that if a contractor's 
performance does not meet the CPL, as determined by the ODR, 
contract actions in addition to payment deductions are 
indicated. These CPLs are to be included in the contracts 
Performance Requirements Summary. [Ref. 19] 
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RSED V3.0 has not been finalized for field use. Prelim- 



inary indications are that it will incorporate one basic 
sampling level, although others may be added. The defect 
rate determined by the sample is adjusted downward by a 
specified "delta” depending on the observed defect rate to 
determine a level appropriate for deductions. The use of 
the delta ensures that the deduction rate will be less than 
or equal to the contractor's actual defect rate 75% of the 
time on a monthly basis, and approximately 90% of the time 
over the annual contract term. The higher confidence level 
associated with the contract term is felt by NAVFAC to be 
the minimum necessary for which to base stronger contract 
actions such as termination or not renewing options. 
Appendix D contains the sample size and delta tables along 
with a sample payment analysis reproduced from NAVFAC 
documentation. [Ref. 19] 

Like the model used for field testing, RSED V3 . 0 uses 
the normal approximation of the hypergeometric distribution 
to determine sample sizes and confidence intervals. The 
relationship used for calculating sample sizes is identical 
to the field model except that different assumptions are 
made. Equation (3) illustrates; 



n 




w 



(P) (1-p) 



+ 




(3) 
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where : 



n = required sample size 

W = .01 (a fixed value of 1% regardless of CPL) 

P = .95 (a fixed value of anticipated performance, 

not CPL) 

Z = .674498 (normal z-statistic for 75% certainty) 

N = estimated population size 

Equation (3) can be rewritten as, 

n = [.00462749 + l]”^ (4) 

N 

The resulting sample size can be seen to be independent 
of the CPL. It will yield an observed defect rate within 1% 
of the actual defect rate 75% of the time, provided the 
actual performance is close to the estimate of 95%. At the 
end of the month, the observed defect rate is used as a 
better estimate of actual performance to calculate a new 
confidence inteirval. This new confidence interval is found 
by solving Equation (1) for "W". The new value for "W" 
becomes the "delta" adjustment. Equation (5) illustrates; 

DELTA = -^) (5) 

where : 

P = obseirved defect rate (ODR) 

Z = 1.28159 (normal z-statistic for 95% certainty) 
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N = 12,000 
n = 732 

The "delta" is calculated with fixed values of "N" and 
"n" so that one table which is a function of the observed 
defect rate may be utilized. This simplification results in 
deltas approximately one percent less than would have been 
obtained using actual sample sizes and populations. It also 
precludes the necessity of having a table for each possible 
sample size and population! The delta in conjunction with 
the ODR can now be used to estimate the true level of 
contractor performance, and to determine a deduction rate to 
be used for extrapolation. The following statements help 
summarize the results of this methodology using an ODR of 
10% as an example: "We are 75% confident that the true 
defect rate is greater than the deduction rate of 8.62% 
(ODR-DELTA) , : or for evaluating performance, "We are 75% 
confident that the true performance level of the contractor 
is less than 91.38% (100% - ODR + DELTA)." 

F. COMPARISON OF THE TWO STATISTICAL BASES 
1 . Inspection Recmirements 

The proposed statistical basis will require greater 
sample sizes to be inspected than the field version. This 
is due to the fact that RSED V3 . 0 seeks to ascertain actual 
contractor performance within a tolerance of one percent. 
Because extrapolation will be applied at all levels of 
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performance under RSED V3.0, this small confidence interval 
is deemed necessary by NAVFAC. Since the field version is 
designed only to determine whether actual performance is 
less than a "tolerable" level, it can be done with a smaller 
sample at a higher level of confidence. 

Another difference between the methods is the 
greater flexibility allowed with the field version. Depend- 
ing on inspector resources and contractor performance, SCMs 
can direct inspection at three confidence levels with large 
differences in inspection requirements. In addition, 
different CPLs can be assigned to further adjust inspection 
requirements within the different levels. However, the 
large influence on inspection requirements exerted by the 
CPL may cause some contract administrators to specify a 
performance level lower than actually desired. Under normal 
levels of inspection, specifying a 95% CPL over a 90% CPL 
results in almost doubling the sample size. The potential 
thus exists for contract performance requirements to become 
a function of inspector resources. 

Increased flexibility could be built into RSED V3.0. 
This would have to be done by allowing different confidence 
intervals, since the confidence level of 75% would seem to 
be approaching a minimum. Specifying larger confidence 
intervals would not change the ODR, which will be the basis 
for contract actions other than extrapolation and remains 
the best estimate of contractor performance. Also, the fact 
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that straight line deductions exist in the Air Force where 
contract specifications have not been met, implies that 
NAVFAC is being unnecessarily conservative. 

2 . Effect On Contractor Performance 

It is reasonable to assume that the contractor will 
be motivated closer to 100% performance under RSED V3.0. 
However, most contract administrators feel that 100% 
performance is unrealistic for most services, and expressed 
the opinion that public works forces rarely performed at the 
now specified CPL. The extrapolation of deductions at 
performance levels which have been previously considered 
satisfactory represent a source of increased risk to the 
contractor which may be expected to result in higher bid 
prices. 

Bid prices may also be higher under RSED V3.0 for a 
more positive reason. The feature of extrapolating at all 
defect levels means that there is a much closer match 
between payment and services received. Contractors will not 
have any incentive to underbid the contract with the goal of 
performing at the CPL and avoiding extrapolation. Thus RSED 
V3 . 0 might result in more bids where the contractor is able 
to make a fair profit at the required performance level. 
Whether or not this results in higher overall bids depends 
on the profit levels now being enjoyed by service 
contractors. 
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There seems to be a greater potential for an adver- 
sarial relationship to develop between the contract 
administrators and contractor under RSED V3.0. This is 
because every observed deficiency would have ramifications 
beyond its own merits. Under the field method, this wasn't 
the case until the CND was reached. At that point there was 
little room for sympathy because the contractor was perform- 
ing at levels well below specified levels. Because the 
quality of services is subject to some inspector judgment, 
borderline cases under RSED V3.0 may be dismissed as 
inspectors carry the increased burden of identifying 
discrepancies which will be the basis for extrapolation. 

3 . Deduction Level 

There are too many variables involved to try and 
quantify the anticipated differences in deductions which 
would be obtained using the two statistical methods. It 
would be fair to say that the potential for higher deduc- 
tions exists under RSED V3.0. The dichotomy remains that 
when either system is functioning as intended, the deduc- 
tions will be at a minimum. 

A major difference between the two methods is the 
treatment of discrepancies outside the random sample (e.g., 
customer complaints, planned sampling) . When deductions are 
extrapolated, it is not correct to also deduct for deficien- 
cies found outside the sample, as these would be 
theoretically included in the extrapolated results. Since 



56 



RSED V3 . 0 extrapolates from the start, it is never 
permissible to deduct for a customer complaint. The field 
method would allow this deduction if the CND had not been 
reached. An interesting result is that in services which 
are customer oriented and a strong customer complaint pro- 
gram exists, extrapolated deductions under RSED V3 . 0 could 
easily be less than what would have been obtained by 
deducting for known deficiencies. This does not appear to 
be a situation that the contractor can take advantage of. 
Even if the services were not amenable to re-work, contract 
actions other than extrapolation could be resorted to. 

4 . Producer/Consumer Risk 

Both the field version and RSED V3 . 0 are biased 
towards the contractor. The field version allows actual 
defect rates to be approximately 1.5 times the CPL before an 
even chance of rejection occurs and straight line extrapola- 
tion is applied. Performance levels under RSED V3 . 0 are 
closely determined, but then a "delta” subtracted from the 
ODR assures that the contractor will be overpaid in the long 
run. 

A normal approach to comparing the producer bias 
inherent in any statistical techniques is the operating 
characteristic curve. However, since there is no reject 
number (CND) involved with RSED V3.0, a curve for it cannot 
be constructed. It is safe to say that in either method, 
the contractor is treated more than fairly. A theoretical 
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even treatment would be to apply the ODR as a deduction rate 
at any performance level. However, NAVFAC is not willing to 
live with the idea that in any one month, the contractor has 
an even chance of being overpaid or underpaid, least not at 
reasonable inspection levels (confidence intervals) . 

5 . Ease of Use 

Both methods appear workable and of comparable 
difficulty to implement and administer. However, under RSED 
V3.0 required sample sizes are specified for much narrower 
population ranges. This increases the potential for 
mismatched samples with actual populations. If clear and 
simple guidance isn't promulgated which defines actions to 
be taken when procedures aren't strictly followed, the 
administrative burden could become excessive. 
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V. SURVEY OF FIELD TESTS 



A. INTRODUCTION 

1 . Purpose 

The purpose of this survey is to identify general 
issues which should be addressed before the extrapolation of 
deductions is approved for general use. Although both of 
the field tests involved family housing maintenance con- 
tracts, the focus of this study remains oriented toward 
general issues. Preferred methods for applying extrapolated 
deductions to any specific contract type are outside the 
scope of this work. 

2 . Background 

The two activities involved in the field test are 
Public Works Center (PWC) Great Lakes and PWC San Francisco. 
Although the housing maintenance requirements of each 
activity are similar, many differences exist regarding both 
the circumstances of the contracting effort and the approach 
taken to employ extrapolated deductions. Key differences 
include CPL requirements, performance weighting, and the 
method for classifying service work for bid and inspection 
purposes . 

Both maintenance contracts were initially written as 
the basis for a CA review. In the case of San Francisco, 
the contract ran for two years before approval to use 
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extrapolated deductions was granted in July, 1985. Prior to 
that time, random sampling had been conducted using the 
procedures of MO-327. 

At Great Lakes, extrapolated deductions was a part 
of the contract from its inception in 1984. NAVFAC provided 
assistance in developing contract specifications, but the 
QAP and related payment documents were generated in-house 
from scratch. The methodology for taking deductions was 
approved at the NAVFAC level before extrapolation was 
allowed. 

3 . Data Collection 

The collection of quantitative data was made diffi- 
cult due to the great amount of paperwork which is generated 
over the course of administrating a service contract. I was 
limited to using payment summary sheets because of the 
fragmentary nature with which relevant information is 
located within that paperwork. These summaries were not 
directly comparable, so some information obtainable at one 
activity could not be collected at another. One example was 
an inability to determine the percent of rework accomplished 
at Great Lakes. This prevented a deduction comparison with 
RSED V3.0 at that activity, which was done with the PWC San 
Francisco data. 

Data collection was focused in both cases on perform- 
ance, inspection and payment history of service work. 
Service work is normally customer generated and requires 
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less than 16 hours to complete. It constitutes the largest 
component of work performed under a housing maintenance 
contract, approximately half. Random sampling was applied 
to service work for the purpose of extrapolating deductions 
at both activities. Other major portions of housing 
maintenance includes change-of-occupancy work and preventive 
maintenance. Data on these categories of work was not 
collected because the change-of-occupancy work is 100% 
inspected and the preventive maintenance data was not easily 
summarized. 

Data collected at PWC San Francisco represented a 
contract period of 11 months. Data collected at Great Lakes 
extended over a 24 month period. In addition to a collec- 
tion of historical data, interviews were conducted with 
contract administrators at both activities. 

B. CONTRACT REQUIREMENTS 

1 . Treatment of Service Calls at Great Lakes 

Service calls at Great Lakes were classified as 

either emergency, urgent, or routine. The implication for 

the contractor was that each required different response 

times. The definitions of each category are given below 

along with the required response time. 

EMERGENCY CALLS — These are problems which pose an immedi- 
ate danger to the occupants or have the potential to cause 
further property damage (e.g., power lines down, overflow- 
ing toilet) . They require contractor response within 30 
minutes on a 24 hour basis. 
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URGENT CALLS — These are problems which do not pose an 
immediate threat to occupants or property, but would soon 
inconvenience and affect the well-being of the occupants 
(e.g. , refrigerator not working, toilet backed up) . The 
contractor is required to respond within two hours of the 
call . 

ROUTINE CALLS — These are problems which cannot be classi- 
fied as either an emergency or urgent situation (e.g., 
tile repair, binding doors) . The contractor is required 
to respond within nine days and to complete the work 
within 14 days. 

The performance of service calls was broken down 
into four performance indicators; responsiveness, quality 
of work, proper classification, and proper accounting. Each 
one of the performance indicators was assigned a weighting 
representative of the "worth" of that portion of the job. 
Performance weights must total 100%. When performance 
indicators are weighted, it means that accomplishment of any 
individual performance indicator earns a percent of the 
service price equal to the weighting. Thus, weighting has a 
direct bearing on the level of deductions to be assessed for 
failure to perform. For example, the performance weighting 
for emergency response is 45%. Suppose that a contractor 
responded to an emergency call one hour late. Regardless of 
the fact that quality work was done and the job was properly 
documented, 45% of the price would be subject to deduction. 

2 . Treatment of Service Calls at San Francisco 



Service calls at San Francisco were also categorized 
as emergency, urgent and routine. However the required 
response times differed as follows; 
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EMERGENCY CALLS — two hours during work hours, four after; 
URGENT CALLS — twenty four hours; 

ROUTINE CALLS — five days response, fourteen days 
completion. 

San Francisco also used the same four performance 
indicators as Great Lakes, but did not assign weights to 
them. In effect, that meant that each one was weighted 
100%. A deficiency in response time, quality, or documenta- 
tion would result in a total deduction being taken. 

Another difference between the activities was that 
San Francisco combined its service calls into three other 
categories for the purpose of bidding and inspection. These 
were interior, exterior and appliances. Thus, exterior work 
might consist of any of the three service call types. 

C. QUALITY ASSURANCE PLANS 

1 . Introduction 

Quality Assurance Plans (QAP) are long, complex 
documents which attempt to provide the activity with a 
structured approach for inspecting all aspects of a contrac- 
tor's performance. It is beyond the scope of this paper to 
attempt to convey the contents of either activity's QAP. My 
intent is to report only those findings which have general 
applicability to RSED. 

2 . General Comments 

A major difference between the two plans was that 
PWC Great Lakes wrote theirs initially to incorporate RESED. 
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San Francisco's plan was based on the MO-327 version of 
random sampling, which did not allow for extrapolation. The 
result was that the QAP in San Francisco had little 
resemblance to the quality assurance program which 
eventually evolved. The fact that PWC San Francisco was 
able to implement a workable program after some trial and 
error, reinforces the wisdom of separating the QAP from the 
formal contract. 

3 . Critical Performance Levels 

The CPLs contained in the QAP were not part of the 
bid information of either contract. NAVFAC's policy then, 
which has just recently changed, is that the CPL was an in- 
house tool only. However, while unknown to the contractor 
during the bidding process, the contractor soon became 
familiar with the CPL as it formed the basis of payment 
analysis and performance judgments. 

The CPL assigned to service contracts in San 
Francisco was 90%, while Great Lakes enforced a CPL of 97%. 
San Francisco felt that a CPL in excess of 90% was too harsh 
and unrealistic. The opinion at Great Lakes was that 97% 
performance was realistic and had been achieved by the 
departing public works force. The SCM at Great Lakes also 
felt that a 97% CPL was high enough to prevent planned 
noncompliance by the contractor. A possibility he thought 
plausible at a CPL level of 90%. 
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The actual weighted average performance or service 
calls achieved at PWC San Francisco was 92.0%, as compared 
to 98.1% at PWC Great Lakes. While contractor performance 
is subject to many variables, including inspector 
aggressiveness, the close parallel between performance 
differences and CPL differences suggests a relationship. 

4 . Performance Weighting 

The failure of PWC San Francisco to weight the 
different aspects of service call performance for payment 
purposes would probably have made extrapolation unsupporta- 
ble in court. This is similar to the Air Force janitorial 
contract [REf. 2]] where a room could be declared unsatis- 
factory and payment denied because an ashtray was not 
emptied. 

Not weighting performance factors also makes bidding 
more difficult and transfers greater risk to the contractor. 
This may have been a factor in PWC San Francisco paying a 
weighted average of $55.00 per service call, while Great 
Lakes paid only $33.00. 

There was not enough data to support conclusively 
whether or not weighting influenced contractor performance 
at Great Lakes (see performance weights in Figure 5.1). 
However, in emergency and urgent calls where responsiveness 
and quality were closely weighted, mean performance was 
within one half of a percent of each other. In the case of 
routine calls, where quality carried twice the weight of 
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responsiveness, quality performance exceeded responsiveness 



by 4.3%. 

D. INSPECTION REQUIREMENTS 

1. Introduction 

This section looks briefly at the use of inspector 
resources at each activity. Any random sampling program 
approved for extrapolated deductions must permit efficient 
use of inspectors, with minimal administrative burden, if it 
is to receive command support. Although many of the inspec- 
tion procedures referred to are in the QAP, all inspection 
related information is presented in this section for 
clarity. 

2 . Sampling Levels 

Figures 5 . 2 and 5 . 3 show the monthly level of 
inspection employed at each activity over the period studied 
along with the population range for each of the service 
categories. It should be noted that due to the lower CPL, 
inspection levels at San Francisco were far below that of 
Great lakes. In fact, although the number of calls at San 
Francisco are higher, they inspected less than half the 
calls inspected at Great Lakes. This relationship would 
suggest too great an incentive for contract administrators 
to sacrifice performance requirements for the sake of saving 
inspector resources. 

These figures also show the potential savings 
possible by combining the services into one group for 
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POPULATION 

MEAN 


POPULATION 

RANGE 


SAMPLE 

MEAN 


INSPECTION 

LEVEL 


ROUTINE CALLS 


536 


391-713 


207 


397. 


URGENT CALLS 


470 


381-604 


163 


357. 


EMERGENCY CALLS 


165 


110-250 


97 


597. 


TOTAL 


1171 


950-1391 


467 


407. 


Figure 5.2 


Inspect! on 


Data at PWC 


Great Lakes 




POPULATION 

MEAN 


POPULATION 

RANGE 


SAMPLE 

MEAN 


INSPECTION 

LEVEL 




INTERIOR CALLS 


961 


814-1070 


88 


97. 




EXTERIOR CALLS 


280 


173-435 


69 


257. 




APPLIANCE CALLS 


192 


139-229 


64 


347. 




TOTAL 


1433 


1252-1669 


22 1 


157. 





Figure 5.3 Inspection Data at PWC San Francisco 
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inspection purposes. If the population of service calls at 
San Francisco were combined into one group of 143 3 calls, 
the required sample size would only be 911. Whether or not 
this could reasonably be considered a homogeneous population 
is subject to debate. Differences in frequency or cost is 
no greater than the variation which occurs within each 
service call category. The economy inherent in large 
populations argues for combining work items where possible. 

A look at the population ranges in Figures 5.2 and 
5.3 illustrates the difficulty in estimating an appropriate 
sample size. Another benefit to increasing the population 
is the greater stability involved. A sample size of 91 
would apply to all populations in the range of 1300-1500. 

3 . Surveillance Level Switching 

Both QAPs provided conditions under which surveil- 
lance levels could be changed. At PWC San Francisco, the 
option was not exercised for two reasons: (1) contract 
administrators felt that reduced inspection levels would not 
have been adequate, and (2) tight surveillance was beyond 
the capacity of the inspector force. 

Surveillance levels were switched as deemed appro- 
priate by the SCM at PWC Great Lakes. Although the guidance 
in the QAP was not strictly followed, the switching tech- 
niqpie was effectively utilized. Even within the routine 
service work category, different surveillance levels were 
used for a period of time. Response time was under tight 
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inspection, while quality of work was inspected under 
reduced surveillance. It should be remembered that the 
incentive for going to reduced inspection was much greater 
at Great Lakes. Even under reduced inspection, at a 97% CPL 
level, greater inspection was required than normal inspec- 
tion at San Francisco. 

The switching of inspection levels does present a 
problem in balancing the workload. The SCM at Great Lakes 
claimed that he was staffed for a normal inspection load. 
When tight inspection was implemented, as it was for the 
first year under routine response, it could be covered only 
through the use of overtime and the shifting of inspectors 
from other assignments. In my view, the only thing gained 
by tightened inspection is the ability to extrapolate the 
same percentages at a higher confidence level and a few more 
observed deficiencies. It would seem more efficient to stay 
at normal inspection for extrapolation purposes and use 
planned inspections where it was necessary to prod the 
contractor a little further. Planned inspections on a 
limited basis can usually be done so that the incidence of 
deficiencies is enhanced. Also, these extra planned 
inspections would not be a firm requirement and would thus 
allow extra flexibility. While deductions resulting from 
planned inspections are allowed only if extrapolation does 
not take place, the rework requirement is still enforceable. 
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4 . Sample Generation 



Both activities made use of a computer program to 
generate random samples. At the beginning of the month, the 
computer would generate the random numbers and these would 
be matched to service call chits as they came in. 

The better program was in use at Great Lakes in that 
it did not require the use of tables (see tables in Appendix 
C) . Required input consisted of a population estimate, CPL, 
surveillance level and lowest sample number. At San 
Francisco only the population estimate and sample size was 
required for input, but determining the appropriate sample 
size required using the tables. Aside from the enhanced 
possibility of making a mistake, the stratification of the 
tables normally meant that required samples were slightly 
greater than if determined directly by formula. In 
addition, the San Francisco program always generated numbers 
starting from one, which meant that chits needed to be 
renumbered every month. 

5 . Sample Adjustment 

The question of sample adjustment arises when the 
actual population of services in a month requires a 
different sample than the one obtained as a result of the 
prior estimate of the population. At PWC San Francisco, 
this possibility was not addressed in the QAP. Lacking 
clear guidance, command procedure has been to evaluate the 
sample as if the population estimate had been correct. This 
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is a simplifying procedure based on conserving inspector 
resources. In evaluating the sampling history, this 
experience has been justified. Due to the relatively large 
samples involved, the lowest being appliances at a mean of 
64 inspections per month, little change in confidence levels 
would have been gained by doing additional sampling. There 
was not time where an adjustment could have resulted in 
extrapolation . 

The procedure written for PWC Great Lakes was more 
sophisticated, yet was unnecessary. At the end of the 
month, the QAE adjusted the sample size using the computer 
program. By giving it the same "seed” number, it 
automatically generated a new sample which either adds a few 
numbers to the previous sample or deletes a few. The QAE 
would then either inspect the additional samples or invali- 
date some already inspected for purposes of extrapolation. 
It is never desirable to invalidate a sample, as this only 
reduces the confidence level of the results. 

As practiced, the SCM decides whether or not to do 
sample adjustment based on the possibility of extrapolation. 
Where the number of defects is close to the CND, he will 
authorize a sample adjustment. Even this procedure is not 
always prudent. Suppose the estimated population for 
emergency calls is 175. At a CPI of 97% and normal surveil- 
lance, this would call for a sample size of 117 and a CND of 
six. If the population turned out to be 200, it would call 
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for a sample size of 127 and a CND of seven. Assuming the 
original number of defects found was six, the inspector 
would be sent out to sample ten more jobs in search of one 
more defect. However, had extrapolation been assessed on 
the basis of the original sample, the unsatisfactory deter- 
mination would have been made at a confidence level of 93%. 
Certainly this is good enough to ascertain unsatisfactory 
performance. Assuming that the original estimate had been 
made in good faith, this extrapolation would no doubt have 
been upheld in court. It would appear that guidance is 
necessary in this area, so that activities can make the most 
intelligent use of their resources. 

E. PAYMENT ANALYSIS 

1. Introduction 

Payment analysis at both activities was based on 
general NAVFAC guidance issued early in 1984 [Ref. 24]. 
Each activity modified the guidance slightly to make it more 
applicable to payment for service call work. This section 
evaluates the methodology of taking extrapolated deductions 
at each activity. The deductions which resulted are then 
analyzed in light of the performance realized. 

2 . Payment Methodology 

Figure 5.4 is an example of payment analysis for 
emergency service calls at Great Lakes. The procedure 
followed at San Francisco is illustrated by a reproduction 
of the payment calculation for exterior maintenance in 
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October, 1985. This calculation is shown in Figure 5.5. 
The obvious difference between the two is that performance 
weighting requires separate analysis for each performance 
indicator at Great Lakes . 

The wording of line "I" instructions (see Figures 
5.4 and 5.5) was found to have caused problems at both 
activities. Line "I", as used at Great Lakes, is identical 
to NAVFAC guidance. However, the confusing instructions 
caused an inspector to not extrapolate deductions when the 
observed defects equaled the CND. 

The instructions associated with line "I” at San 
Francisco leaves no clue as to what the actions should be 
when the observed defects equal the CND. In the one 
instance where the situation occurred, deductions were not 
extrapolated when the observed defects equaled the CND. 
Contract administrators were unaware of the proper procedure 
to follow in this case. Clearer instructions should advo- 
cate extrapolation when the, "observe defects are equal to 
or greater than the CND." 

3 . Treatment of Licmidated Damages 

Liquidated damages were extrapolated at both 
activities (see Figures 5.4 and 5.5). The SCM at Great 
Lakes justifies this action by claiming that the customer 
suffers damages through the failure of the contractor to 
perform. This issue remains unresolved. In their new 
contracts, San Francisco will continue to extrapolate 
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EXTERIOR SERVICE CALL DEDUCTIONS 


A. 


COST OF REQUIRED SERVICES 


$17, 809.09 


B. 


TOTAL UNITS OF WORK 


262 


C. 


COST PER UNIT OF WORK 


$64. 94 


D. 


CRITICAL PERFORMANCE LEVEL (CPL) 


907. 


E. 


CRITICAL NUMBER OF DEFECTS 


12 


F. 


SAMPLE SIZE 


Zl 


G. 


NUMBER OF OBSERVED DEFECTS 


15 


H. 


PERCENT OF DEFECTIVE UNITS IN SAMPLE 


20 . 37. 


I. 


DEDUCTION FOR DEFECTIVE UNITS 






<1) If G is less than E (GXC) 


NA 




(2) If G is more than E (HXA) 


$3, 704.29 


J. 


DEDUCTION FOR LIQUIDATED DAMAGES (1X107.) 


$370.43 


K. 


NUMBER OF RANDOM SAMPLE UNITS REWORKED 


15 


L. 


PAYMENT FOR REWORK (KXC) 


$974. 10 


M. 


NUMBER OF VALIDATED CUSTOMER COMPLAINTS 


7 


N. 


COST OF CUSTOMER COMPLAINTS (MXC) 


$454.58 


0 . 


LIQUIDATED DAMAGES FOR CUST COMP (NX 107.) 


$45. 46 


P. 


CUSTOMER COMPLAINTS REWORKED 




Q. 


PAYMENT FOR REWORK 


$129.88 


R. 


DEDUCTIONS FOR CUST COMPLAINTS (N-O-Q) 


$370. 16 


S. 


TOTAL DEDUCTIONS (I+J+O+R) 


$4. 444. 88 


T. 


PAYMENT FOR SERVICES (A-S) 


$13.364. 21 



Figure 5.5 Exterior Service Payment at PWC San Francisco 
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liquidated damages, while Great Lakes will claim liquidated 
damages only for work which is re-inspected. 



A firm policy needs to be established for each 
category of work. Where a government function is the recip- 
ient of the service, as in a janitorial contract, extrapo- 
lated damages may be more easily supported. This is a 
different situation than attempting to assess damages for 
the housing occupants' inconvenience. In any case, the 
extrapolation of liquidated damages can be used as extra 
incentive to obtain contractor compliance with the contract. 

4 . Treatment of Validated Customer Complaints 

Both activities took deductions for validated 
customer complaints in addition to extrapolation. This 
procedure is incorrect. The extrapolation of deductions 
theoretically accounts for all deficiencies which exist 
outside the sample. The most advantageous way to treat 
validated customer complains under the field version is to 
take deductions unless the CND has been reached. As can be 
seen in Figure 5.5, PWC San Francisco makes no such 
distinction. 

5 . Deduction Data 

Extrapolation occurred ten times at PWC Great Lakes 
and only once at PWC San Francisco, although each lost an 
additional opportunity to extrapolate when the observed 
defects equaled the CND. The difference can be attributed 
mainly to the lower CPL used at San Francisco. However, 



77 



it's the potential of extrapolated deductions that is being 
counted on to improve contractor performance. Greater 
deductions are not being sought for their own sake. This 
potential is evident by analyzing the nature of the 
deductions . 

A total of $19,195.00 was taken in service call 
deductions over the 24 months evaluated. That represents 
2.1% of the contracted price for these services. Forty-five 
percent of those deductions, $8,694.00, were attributable to 
extrapolation beyond observed defects. 

While only one incident of extrapolation took place 
in eleven months at San Francisco, it resulted in deductions 
of $2,731.00 above the observed deductions. That represent- 
ed 23% of the total deductions of $11,768.00. Total 
deductions at San Francisco amounted to a reduction in the 
contract price of 1.3%. 

Thus it can be seen that a contractor certainly has 
reason to avoid extrapolation. When only a small portion of 
a contract is being inspected the number of observed defects 
will also be small. Therefore, deductions taken for 
observed defects will be minor. Where the contractor can 
also expect to rework the item, there is little financial 
incentive to do it right the first time. However, if 
deductions are extrapolated, the final percent deducted will 
more closely reflect actual performance. The impact of this 
is shown graphically in the next section. 
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6 . 



Effect on Performance 



It is impossible to definitively state what factors 
influenced performance at each activity. One fact is that 
both contractors quickly learned to avoid extrapolation. 
The ten incidences of extrapolation at PWC Great lakes 
occurred in the first eleven months of the contract. Seven 
of those extrapolations were assessed for unsatisfactory 
performance in response to routine calls. Response to 
routine calls was the only area where Great Lakes experi- 
enced any problems. 

Figure 5.6 plots contractor performance in response 
to routine calls at Great Lakes, along with the correspond- 
ing percent of contract payment each month. Extrapolation 
took place from June to October; and then again in December 
and February. It should be noted that in these months 
payment actually fell below the performance level. This is 
an undesirable state of affairs from NAVFAC's point of view, 
and occurred because deductions were taken for validated 
customer complaints in addition to the extrapolation. Due 
to the high GPL of 97%, the contractor was forced to perform 
within a relatively narrow band to avoid extrapolation. In 
February of 1985, for example, the CND was exceeded even 
though performance was at 95.4%. The last year of the con- 
tract has seen steady high performance in all service areas. 

Performance at PWC San Francisco was erratic in all 
service categories. Figure 5.7 plots performance in 
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MOHTHS 



Figure 3.6 Perf ormance/Payment Comparison at PWC Great Lakes 




Figure 3.7 Perf ormance/Payment Comparison at PWC San Francisco 
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exterior maintenance, along with the corresponding percent 
of contract payment each month. Except for the extrapola- 
tion which occurred in the first month, performance in 
exterior maintenance typified the other two work categories. 
The potential of extrapolated deductions is most evident 
from this plot. It can be seen that where extrapolation 
took place, payment was well below performance. Again, this 
is a result of taking deductions outside the sample. 
However, due to the low CPL of 90%, the contractor was able 
to perform at levels as low as 85.7% without exceeding the 
CND. Because the level of inspection was low, large gaps 
existed between payment and performance. In fact, the 
closer the contractor could get to the CND without reaching 
it, the greater would be his favorable gap between perform- 
ance and payment. Thus, it appears that the contractor was 
able to manipulate his performance to maximize payment 
differential . 

7 . Comparison with RSED V3 . 0 

I did a deduction analysis with the PWC San 
Francisco data assuming that RSED V3.0 had been in affect. 
The questionable assumption in such an analysis was that 
contractor performance remained the same. This is qpaestion- 
able because performance would have been judged unsatsifac- 
tory eight times vice once, resulting in more pressure 
through CDRs. In addition, the fact that payment would have 
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more closely matched performance would have provided 
additional incentive. 

The level of deductions under RSED V3 . 0 would have 
been $3,766.00 per month. This represents 4.7% of the 
contract price and a monthly increase in total deductions 
over the field method of $2,696.00. This increased deduc- 
tion level would have to be paid for with increased inspec- 
tions . Monthly inspections required by RSED V3 . 0 would 
represent an 81% increase from 221 inspections to 401. At 
least one new inspector would have been required to cover 
this requirement. The extra deductions would pay for that 
new inspector only if performance did not increase! 

Discrepancies between pay and performance will be 
minimal at all levels under RSED V3.0. Payment will 
normally be about 1% in excess of performance if this 
methodology is properly applied. This will eliminate any 
motivation for the contractor to underperform, as now exists 
at San Francisco. However, the fact that all deductions 
will be extrapolated removes a clear goal tied to the 
pocketbook. That "goal” appeared effective in ensuring 
contractor performance within a narrow band at PWC Great 
Lakes . 
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VI . CONCLUSIONS AND RECOMMENDATIONS 



A. GENERAL ISSUES 

1. Need for Oversight 

This study found that in a few areas, discrepancies 
existed between the RSED procedures implemented at the test 
activities and the intentions of NAVFAC. These discrepan- 
cies occurred because of either a lack of clear guidance or 
an omission on the part of the activity. Major examples 
indicating a lack of communication between NAVFAC and the 
activities are: (1) the failure of both activities to 
extrapolate when observed defects equaled the CND, (2) the 
taking of deductions in addition to extrapolation, and (3) 
different approaches planned for the extrapolation of 
liquidated damages. 

Problems such as these must not only be anticipated 
in future guidance, but incorporated into sample working 
documents where possible. In addition, the potential for 
misunderstandings justifies an internal command assistance 
program. Evaluation of RSED plans at the EFD level would 
promote uniformity and provide assistance. This simple 
followup would help preclude the Air Force experience with 
GAO audits. Because of the small number of activities 
involved, effective oversight would not be difficult to 
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implement. When correct procedures became entrenched in the 
system, this special effort could be phased out. 

2 . Treatment of Sample Adjustment 

The review of activity QA plans and discussion with 
contract administrators indicated that the potential for 
unnecessary sample adjustment was great. Current guidance 
does not clearly address the issue of sample size adjust- 
ment. This problem becomes even more critical under RSED 
V3 . 0 due to the monthly extrapolations and narrow sampling 
requirements. Lacking statistical backgrounds, contract 
administrators' actions are unpredictable. Worse than the 
expedient procedure at PWC San Francisco, would be to 
overcompensate . 

The analysis of the effect that sample size had on 
confidence level suggests that original samples should 
provide adequate basis for contract action in all but the 
most extreme cases. A simple cutoff might be a minimum 
sample size of 50. Where the original sample is based on a 
good faith estimate and exceeds 50, no adjustment would be 
necessary. Specifying a minimum confidence level of 50% 
before adjustment was necessary would be another approach. 
However, that would require software support to speed the 
process and ensure against miscalculation. 

3 . Combining Homogeneous Populations 

The benefits of random sampling increase exponen- 
tially with population. Due to the nature of service 
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contracting, strict homogeneity is almost never achieved 
within any population. Where two services have the same CPL 
and similar unity costs, the possibility of combining those 
populations should be examined. Inspection to different 
standards would not present a problem. This process would 
be similar to the combination of service calls into another 
category as was done at San Francisco. However, in that 
case no benefits were realized because three categories were 
still used. Candidates for combination should be studied 
and methods promulgated to make RSED more efficient. 

4 . Surveillance Levels 

The wisdom of sampling at increased levels of 
inspection is questionable. Tightened inspection does not 
result in increased levels of extrapolation. Sample sizes 
required by tightened inspection (99% confidence) are signi- 
ficantly higher than normal sampling. It was done at PWC 
Great lakes through the use of overtime and reassignment. 
At PWC San Francisco, constrained resources prevented its 
use. 

Where contractor performance has been poor and 
inspector resources are available, minimum levels of random 
sampling should be supplemented with planned sampling. This 
procedure makes use of inspector judgment to enhance the 
probability of identifying defects while maintaining 
extrapolation. More flexibility is gained because there is 
no need to obtain a "proper" sample at the higher level. 
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Future guidance should clearly document the procedures for 
mixing these two QA techniques and promote its use. 

5 . Performance/Pavment Match 

Under the field version, the mismatch between 
payment and performance increases with a lower CPL. While 
this mismatch provides a powerful incentive to avoid 
extrapolation, it appeared to result in planned nonconform- 
ance at PWC San Francisco. Deductions can only act as a 
motivator where payment closely follows performance. 

Guidance specifications, when developed, should 
include minimum CPLs. This is especially true where the 
field version is in use, as inspection levels are a function 
of the CPL. Activities should be required to justify lower 
CPLs. This study indicates that a CPL of 90% for service 
calls is too low. 

6 . Computer Assistance 

The complexity of RSED requires computer support 
over a range of operations. This support is necessary to 
both reduce the administrative burden and minimize the 
chance of error. Programs now exist which are close to 
satisfying this need. Full service programs should include: 
sample generation with flexible numbering, sample adjustment 
with decision routine, payment analysis, and sample history 
if appropriate for identifying planned samples. 

Computer support should be a requirement for activi- 
ties planning on implementing a RSED program. The high 
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payback of random sampling would justify this investment. 
Software issued by NAVFAC could also be designed to help 
ensure uniformity in the application of RSED. 



B. RSED V3.0 

The proposed method of taking extrapolated deductions is 
an improvement over the field version in that deductions and 
inspection requirements are not a function of the CPL. It 
provides a better match between payment and performance at 
lower CPL levels. The sample sizes required by RSED V3 . 0 
are approximately equal to normal sampling at a CPL of 95% 
and reduced sampling at 97%. Thus, activities with a 
planned CPL of 90% or less would experience a significant 
increase in sample requirements. 

Possible disadvantages associated with RSED V3.0 
include: 

(1) RSED places more emphasis on proper samples due to 
monthly extrapolation. 

(2) There is no clear measure of good performance which 
provides financial incentive. 

(3) Where contractor performance is consistently good, 
there is no reduced inspection level possible because 
of the need to extrapolate. 

(4) Influence of customer complaints is diminished 
because they cannot be used to take deductions. 

(5) The addition of a payment "adjustment” is one more 
complication added to the payment analysis. Rounding 
procedures must be clearly addressed. 

The importance of these disadvantages was not studied. 
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I support NAVFAC's intentions to try RSED V3 . 0 because 
of the weakness of the field version in influencing 
performance at low CPL levels. This phenomenon was demon- 
strated at PWC San Francisco. However, results from the 
field will have to be analyzed before a definitive statement 
can be made. Interesting comparisons could be made by 
shifting PWC San Francisco and PWC Great Lakes to RSED V3 . 0 
under the same contractor. 



C. RECOMMENDATIONS FOR FUTURE STUDY 

1. The impact of RSED V3 . 0 on contract price and 
performance. 

2. Relationship between inspector aggressiveness and QA 
technique. For example, if an inspector knows that 
every defect identified will have ramifications beyond 
its own importance, will he be less likely to document 
borderline cases? 

3. A cost/benefit study of RSED versus planned sampling. 

4. A legal study which attempts to define the parameters 
within which NAVFAC must stay when designing a RSED 
program . 

5. Identification of specific contractors which serve 
both the Navy and Air Force under their respective 
RSED programs. Compare payment and performance. 
Interview the contractor for his perspective. 
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APPENDIX A 



USING MIL-STD-105D 

The procedure contained in this appendix is taken in 
part from OFPP Pamphlet No. 4, "A Guide For Writing and 
Administering Performance Statements of Work For Service 
Contracts." The OFPP approach is equivalent to Air Force 
inspection procedures given in AFR 400-28. A sample size 
and corresponding accept/reject numbers are then determined 
for a hypothetical case, along with the associated 
producer/consumer risk. 
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SAMPLING PLAN 



A. Deciding on the Acceptable Quality Level (AQL) . The AQL 
is the highest number of defects per hundred, highest 
percent defective or highest number of defects that can be 
allowed for any service performance indicator. There are 
only a limited number of AQLs listed in MIL-STD-105D but, in 
virtually all cases, one will be close enough to control the 
contractor's level of service. 

(1) The first step in designing a sampling plan under 
MIL-STD-105D is the selection of a realistic AQL. No 
service can be perfectly performed. 

(2) Find the closest AQL from Figure 1 and use it to 
replace the original AQL on the Performance Requirement 
Summary. For example, the AQL for taxi service might have 
been 5 percent. This would be changed to 4 percent or 6.5 
percent since 5 percent does not appear in the figure. 

B. Determining the Lot Size. To determine the sample size, 
the lot size must be known. The lot is how often the 
contractor provides the service in a period of time. 

(1) To determine the lot size, estimate the frequency 
of the service to be sampled, during the period it is to be 
sampled. 

(2) In the case of workorders, the monthly lot size can 
be estimated from historical information on file. 

C. Determining the Sample Size. Use Figure 2 to identify 
an appropriate sample size for a given lot size. 

(1) Use the normal sample size column unless there is a 
limited number of QAEs or unless the cost of an inspection 
suggests the use of the medium or small sample size column. 

(2) Use the medium or small sample size, if inspec- 
tions for a particular service are lengthy or hinder the 
contractor's ability to provide service to customers. 

D. Selecting the Rejection Level. Use MIL-STD-105D to 
identify the acceptance and rejection level for the sample 
size (see Figure 3) . To use the figure, begin with the 
known values for the AQL and the sample size. 

(1) Find the selected sample size (in the sample size 
column) and read across that line to the column for the 
selected AQL. At that point there will either be two 
numbers or an arrow pointing up or down. 
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AT leviable Acceptable Quality Levels 



0.010 z 


1.0 z 


0.015 S 


1.5 Z 


0.025 : 


2.5 Z 


0.040 : 


4.0 Z 


0.065 Z 


6.5 Z 


0.10 z 


10. Z 


0.15 Z 


15. Z 


0.25 Z 


25. Z 


0.40 Z 


40. Z 


0.65 Z 


63. Z 



Figure 1. List o-f MIL-STD-105D Acceptable .Quality Levels 
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Lot Size 


Normal Sample 
Size 


Medium Sample 
Size 


Small Sample 
Size 


2-3 


2 


2 


2 


9-15 


3 


2 


2 


16-25 


5 


3 


3 


26-50 


8 


5 


5 


51-90 


13 


5 


5 


91-150 


20 


8 


8 


151-280 


32 


13 


13 


231-500 


50 


20 


13 


501-1,200 


80 


32 


20 


1,201-3,200 


125 


SO 


32 


3,201-10,000 


200 


80 


32 


10,001-35,000 


315 


125 


50 


35,001-150,000 


500 


200 


80 


150,001-500,000 


SOQ 


315 


80 


500,000 and over 


1250 


500 


125 



Figure 2. Sample Sizes 
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Figure 3. MIL-STD-105D Normal Sampling Table 



(2) If there is an arrow, follow the direction of the 
arrow until it leads to a pair of numbers. Of the two 
numbers at the intersection or at the end of the arrow, the 
number on the left (Ac or accept) indicates the maximum 
number of defects which can occur in a sample and still 
permit the total group or lot to be judged acceptable. 

(3) When there is no accept or reject number for a 
given sample size and AQL, following the arrow will also 
cause a change in sample size. For example, with an AQL of 
1.5 and a sample size of 20, the sample size would become 
32. 



(4) The number on the right (Re or reject) indicates 
the minimum number of defects that occur in a sample which 
causes the total group or lot to be judged unacceptable. 
For example, suppose the sample size is determined to be 32 
and the AQL has been set at 6.5 defects per hundred. Find 
the number 32 in the sample size column and read across that 
line until the AQL column for 6.5 has been reached. The two 
numbers at that intersection are 5 and 6 . 
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EXAMPLE CALCULATION 



Given information: 

1. Estimated number of services for month: 1000 

2. AQL: 6.5 percent 

3 . Normal inspection desired 

Determined from the tables: 

1. Sample size: 80 

2. Accept number: 10 

3. Reject number: 11 

It should be noted for this example that even though the 
activity has specified 6.5% defective as satisfactory 
performance, it will accept 10 defective work units out of a 
sample of 80!. This equates to 12.5% of the sample being 
defective. 

From Figure 4 it can be seen that if the actual level of 
defects present were 8.0%, the probability of acceptance 
would be 95%. This equates to a producer (contractor) risk 
of 5%. At the same time, even if the actual level is as 
high as 11%, it will still be accepted 75% of the time. 
Thus consumer (government) risk at that level of performance 
is 75%. This represents a large bias in favor of the 
contractor. 
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CHART C-O • OPERATING CHARACTERISTIC CURVES FOR NORMAL-TIGHTENED SINGLE SAMPLING SCHEMES 
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APPENDIX B 



MO-327 RANDOM SAMPLING PROCEDURES 



This appendix is an excerpt from the current NAVFAC MO- 
327 entitled "Service Contracts: Specifications and 
Surveillance." It represents the NAVFAC guidance for random 
sampling prior to the field testing of extrapolated deduc- 
tions. The MO-327 is presented here to illustrate the 
limited random sampling guidance available to activities 
planning to begin a random sampling program. Along with an 
explanation of random sampling, this appendix includes 
inspection tables and a procedure to generate random 
numbers . 

When NAVFAC has finalized their procedure for implement- 
ing extrapolated deductions Navy-wide, the guidance will 
appear in a new MO-327. 
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RANDOM SAMPLING 



1. GENERAL . Random sampling is a surveillance method based 
on statistical theory. The key element of random sampling 
is that each and every occurrence of work has an equal 
chance of being evaluated. In order to achieve the desired 
end results, knowledge of the Contractor's overall 
performance based on evaluation of only part of the work, 
surveillance by random sampling must be applied properly. 
Key elements of random sampling are: 

a. Sample size is specified for a given population to 
achieve a predetermined level of statistical accuracy. 

b. The sample of work occurrences selected for 
evaluation must be selected by a random process in which 
each occurrence has equal chance for selection. 

c. Once an evaluation schedule has been established, it 
must be followed through the surveillance period (i.e., 
monthly schedules) . 

d. Surveillance data gathered by other methods (i.e., 
customer complaints and unscheduled inspections) can not be 
combined with data gathered by random sampling. 

e. Assessment of the Contractor's overall performance, 
projected from the observed condition of the sample, will 
always have the potential to be in error. Statements as to 
overall performance should be stated as "The Contractor's 
overall defect rate is in excess of X % . " (Where "X" 
equals the observed defect rate minus one half the AQL — ex. 
ODR (9.2%) - 1/2 AQL (10%) = 4.2%.) 

2. MECHANICS OF RANDOM SAMPLING . Random sampling is a 
structured approach based on statistics to contract 
surveillance. As such, there is a set procedure in its 
application. The mechanics of applying random sampling are 
as follows. 

3. POPULATION . The total number of work occurrences for a 
given function that are to be performed during the surveil- 
lance period must be known or accurately estimated. The 
Inventory of Service Worksheet is used to determine popula- 
tion size. 

a. When work is scheduled, population size is easy to 
determine. 

Example: Activity X has 80 dumpsters, 70 are 
emptied weekly and 10 are emptied twice a week. The 
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population, total number of work occurrences per month, is 
360 (70 X 4) + (10 X 4 2) . 

b. When work is unscheduled, population size must be 
estimated. 

Example: The number of service requests for the 

past six months were; 

Jan 321 
Feb 301 
Mar 295 
Apr 337 
May 340 
Jun 320 



Total 1914 

The average number of service calls per month has been 319 
(1914/6) . This would be the expected population for service 
calls for next month unless there is some known reason to 
expect a change. 

4. SAMPLE SIZE . Sample size requirements are based on AQL, 
population size, and level of surveillance. Sample size 
tables are used to determine sample size. Tables for normal 
surveillance, reduced surveillance, and increased surveil- 
lance are attached. 

a. Select the table with the desired level of surveil- 
lance (Tables I, II, or III) . 

b. Select the column with the required AQL (.05, .10, 

. 15 , . 20 , or . 25) . 

c. Select the row that is closest to the population 
size, preferably the next largest entry. 

d. The number indicated by the row and column selection 
is the sample size required for surveillance in one surveil- 
lance period. 

(1) Daily surveillance requirements will be deter- 
mined by dividing the required period's sample size by the 
number of days that surveillance is to be conducted. 

(2) Weekly surveillance requirements will be 
determined by dividing the required sample size by the 
number of weeks in a period. 

(3) When computing weekly or daily sample sizes, 
always round up to the next whole number (e.g., 45 monthly 
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samples required and 20 work days per month results in a 
daily sample size of 3 - (45/20) = 2.25 and rounded up 
results to 3) . 

5. Sample Selection . The final thing to be decided in 
sampling is how the sample will be drawn. The objective in 
the method is to insure that the sample is random (that is, 
that all services have an equal chance of being selected) . 
To achieve random selection, use a random numbers table as 
explained in the following examples. (A random numbers 
table. Table IV, is attached) . Most items will fall into 
one of these examples. 

a. Use of The Random Numbers Table . The random numbers 
in Table IV are arranged in groups of two. 

(1) To use the table, begin by picking at random a 
group of numbers on any page of the table. This is usually 
done by closing the eyes and pointing with a pencil or 
finger to some initial group. 

(2) To identify additional random numbers, follow a 
pattern. Go along a given line to its end and then along 
the next line to its end and so on through the table until 
enough numbers have been selected or until the table ends. 

(3) If the table ends and there are still more 
numbers to select, go back to the beginning of the table and 
continue using the same pattern. Use various patterns 
alternately, for example, use lines for one sample, use 
columns for the next sample, and use a diagonal pattern for 
the third sample. 

b. How To Use the Random Numbers Table To Identify a 
Random Sample of Consecutively Numbered Work orders . 
Suppose one has to identify a random sample of 97 work 
orders for evaluation. (Sample size is based on a 
population of 319 using normal surveillance.) This can be 
done at the beginning of the month (before the work orders 
are written) or at the end of the month. 

(1) If there are, or might be, 319 consecutively 
numbered work orders to select from, then one begins by 
listing the lowest work order number (known or projected) . 
This cold be $001, or possibly 443, or any other sequen- 
tially assigned number. List the highest work order number 
(known or projected) ; in this case, it could be #319 or 762. 
For this example, use work orders numbered #443 to 762. 

(a) Select 97 three digit numbers from Table IV 
using a consistent pattern. 
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(b) If random numbers selected are not between 
443 and 762, discard the number outside the designated range 
and select a new number. 

(2) For example, using the initial entry on Table 
IV we would select number 441. This number is too low. The 
next number, going down the column, is 343 again too low. 
The third number selected is 749. This number falls in the 
range of work order number (443-762) subject to inspection. 
So work order number 749 is selected to be inspected. The 
next work order number selected is 523. 

(3) This process would be continued until three 
work orders are selected. 

c. How To Use The Random Numbers Table To Identify A 
Random Sample From A Group of Items . If a number of items 
need to be sampled that are not consecutively numbered, the 
simplest solution is to list the identifiers, for all the 
items in a column, on a piece of lined paper. 

(1) Next, number the lines consecutively, beginning 
with the number one. Now use the random number table to 
draw the sample from the line numbers. A selected line 
number leads to the identifier located on that line, and 
that identifier tells which item to sample. For example, if 
one chooses to sample a set of work orders with attached 
sales slips, one is not going to have to have a set of 
consecutively numbered work orders because not every work 
order has a sales slip attached. 

(2) List the work orders with sales slips in a 
column, number each line in the column, and randomly select 
enough line numbers to make up the sample. 

d. How To Use The Random Numbers Table To Identify a 
Random Samples of Days . Suppose one wants to identify four 
days in the month on which to sample something. The days of 
the month can be numbered 01 to 31 (or less, as 
appropriate) . 

(1) It is best to use a starting point different 
from the one used in the previous example. For the purpose 
of this example, it is being used again. 

(2) One can move down the column from number to 
number until the first number between 01 and 31 is spotted. 
In this case, it is 22. Thus the 22nd day of the month is 
selected for sampling. 

(3) Continuing in this fashion, one discovers that 
11 is the next number selected. This number is disregarded. 
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Proceed in this manner until the four days for sampling have 
been identified. In our example, the 4 days selected would 
be 22, 11, 10 and 24. 

(4) If it is not desirable to sample on weekends, 
discard those days selected that happen to fall on a weekend 
and continue that selection until the proper number of days 
has been selected. 

e. How To use the Random Numbers Table To Identify a 
Random Sample of Times of Dav . If one wants to select 
random times of day to sample a service such as taxi or bus 
service, use the 24 hour clock. 

(1) If there are any constraints during each 24- 
hour period, take them into consideration. For example, 
suppose that base bus service operates between 0700 and 
2300. Convert these times to minutes (e.g., 0700 = 0, 0410 
= 13 0, 1215 = 315, etc.). Again, using Table IV and 
selecting three digit numbers and proceeding across the line 
from the initial number, one comes to 441, or 1421 hrs, as 
the first random time. 

(2) The next random number is 34 3, or 1243 hrs. 
The number is good and so one schedules an observation for 
1243 hrs. 

(3) Proceed in this manner until the desired number 
of sample times have been identified. 

f . How To Insure Variety in the Use of the Random 
Numbers Table . The use of variety in the random number 
table ensures that detectable patterns do not occur. 

(1) Success in using the tables requires 
consistency but also variety. The above information should 
ensure that the tables are properly used and that the sample 
is randomly drawn. 

g. Other Random Numbers Generating Methods . The use of 
a hand held calculator with a random number generating capa- 
bility is an alternative to the use of random numbers table. 
Using this type of calculator the QAE would enter the 
minimum value and maximum value and numbers generated would 
always be within the desired range. 
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SAMPUe SIZE TOR RJUXK3J SOTV2ILLANC2 

iSUTCrZZLLMiCZ LZ7ZL I) 



POPCTLATION AQL 



s::z 


.05 


.10 


.15 


.20 


.25 


so 


31 


21 


16 


11 


9 


75 


34 


25 


17 


13 


10 


100 


44 


27 


18 


13 


10 


125 


49 


28 


19 


14 


10 


150 


52 


30 


20 


14 


10 


175 


55 


30 


20 


14 


10 


200 


57 


31 


20 


14 


11 


225 


59 


32 


20 


14 


11 


250 


60 


32 


21 


14 


11 


275 


61 


32 


21 


15 


u 


300 


63 


33 


21 


15 


11 


325 


64 


33 


21 


15 


u 


350 


64 


33 


21 


15 


u 


375 


65 


33 


21 


15 


11 


400 


66 


34 


21 


15 


u 


425 


67 


34 


21 


15 


u 


450 


67 


34 


21 


15 


11 


475 


63 


34 


21 


15 


u 


500 


68 


34 


21 


15 


u 


550 


69 


34 


22 


13 


11 


600 


70 


34 


22 


15 


11 


650 


70 


35 


22 


15 


11 


700 


71 


35 


22 


15 


11 


750 


71 


35 


22 


15 


11 


300 


72 


35 


22 


15 


11 


850 


72 


35 


22 


15 


11 


900 


73 


35 


22 


15 


11 


950 


73 


35 


22 


15 


11 


1000 


73 


35 


22 


15 


11 


1100 


74 


35 


22 


15 


u 


1200 


74 


35 


22 


15 


11 


uoo 


74 


36 


22 


15 


u 


1400 


75 


36 


22 


15 


11 


1500 


75 


36 


22 


15 


u 


1600 


75 


36 


22 


15 


u 


1700 


75 


36 


22 


15 


u 


1800 


76 


36 


22 


15 


11 


1900 


76 


36 


22 


15 


u 


2000 


76 


36 


22 


15 


11 


2500 


76 


36 


22 


15 


11 


3000 


77 


36 


22 


15 


11 


3500 


77 


36 


22 


15 


11 


4000 


77 


36 


22 


15 


11 


4300 


77 


36 


22 


15 


11 


5000 


78 


36 


22 


15 


11 


6000 


78 


36 


22 


15 


11 


7000 


78 


36 


22 


15 


11 


3000 


78 


36 


22 


15 


11 


9000 


78 


36 


22 


15 


11 


10000 


78 


36 


22 


15 


11 
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SAMPLZ SIZZ FOR NORMAL SURVEILLANCZ 

(SORVEILLANCZ LZVSL II) 



POPULATION AQL 



SIZ2 


.05 


.10 


.15 


.20 


,25 


SO 


43 


37 


32 


27 


23 


75 


60 


49 


40 


32 


27 


100 


76 


sa 


46 


37 


29 


12S 


a9 


66 


51 


40 


31 


150 


101 


72 


54 


42 


32 


175 


ill 


78 


57 


43 


33 


200 


121 


82 


60 


45 


34 


225 


129 


86' 


62 


46 


35 


250 


U7 


89 


63 


47 


35 


275 


144 


92 


65 


48 


36 


300 


151 


95 


66 


48 


36 


325 


157 




67 


49 


37 


• 350 


162 


99 


68 


49 


37 


375 


167 


101 


69 


SO 


37 


400 


172 


103 


70 


50 


37 


425 


176 


105 


71 


51 


38 


450 


lai 


106 


71 


51 


38 


475 


ia4 


107 


72 


51 


38 


500 


laa 


109 


72 


52 


38 


550 


195 


111 


73 


52 


38 


600 


201 


U3 


74 


52 


39 


650 


206 


114 


75 


52 


39 


700 




U6 


75 


53 


39 


750 


215 


117 


76 


53 


39 


aoo 


219 


ua 


76 


54 


39 


350 


222 


119 


77 


54 


39 


900 


226 


120 


77 


54 


39 


950 


229 


121 


7a 


54 


39 


1000 


231 


122 


7a 


54 


39 


1100 


236 


123 


78 


55 


40 


1200 


241 


124 


79 


55 


40 


1300 


244 


125 


79 


55 


40 


1400 


243 


126 


80 


55 


40 


1500 


251 


127 


ao 


55 


40 


1600 


253 


12a 


ao 


55 


40 


1700 


256 


12a 


ao 


35 


40 


laoo 


25 a 


129 


81 


56 


40 


1900 


260 


129 


ai 


56 


40 


2000 


262 


uo 


ai 


56 


40 


2500 


269 


131 


a2 


• 56 


40 


3000 


274 


132 


82 


56 


41 


3500 


277 


133 


82 


56 


41 


4000 


2ao 


134 


83 


57 


41 


4500 


2a2 


134 


83 


57 


41 


5000 


284 


135 


33 


57 


41 


6000 


2a7 


135 


33 


57 


41 


7000 


289 


136 


83 


57 


41 


3000 


290 


136 


84 


57 


41 


9000 


291 


136 


84 


57 


41 


.3000 


292 


137 


84 


57 


41 
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SAMPLE SI2Z REQaiRSMENTS TOR INCREASE) SURVEILLANCS 
(SUHVEILLANCS LEVEL III) 



POPULATION AQL 



SIZE 


.05 


.10 


.15 


.20 


.25 


50 


47 


43 


39 


35 


31 


75 


67 


59 


52 


46 


39 


100 


86 


74 


63 


54 


45 


125 


104 


87 


72 


60 


50 


150 


121 


98 


80 


65 


53 


175 


136 


108 


86 


69 


56 


200 


151 


U6 


92 


73 


58 


225 


164 


124 


97 


78 


60 


250 


177 


132 


101 


79 


62 


275 


189 


U8 


105 


81 


63 


300 


201 


144 


108 


83 


64 


325 


211 


150 


in 


85 


65 


350 


222 


155 


U4 


86 


66 


375 


231 


159 


U6 


88 


67 


400 


241 


164 


U9 


89 


68 


425 


249 


168 


121 


90 


69 


450 


258 


172 


U3 


91 


69 


475 


266 


175 


124 


92 


70 


500 


273 


178 


126 


93 


70 


550 


288 


184 


129 


95 


71 


600 


301 


189 


132 


96 


72 


650 


313 


194 


134 


97 


72 


700 


324 


198 


136 


98 


73 


750 


334 


202 


138 


99 


74 


800 


344 


206 


139 


100 


74 


850 


352 


209 


141 


101 


75 


900 


361 


212 


142 


101 


75 


950 


368 


214 


143 


102 


75 


1000 


376 


217 


144 


103 


76 


UOO 


389 


221 


146 


103 


76 


1200 


401 


225 


148 


104 


77 


1300 


4U 


228 


149 


105 


77 


1400 


421 


231 


150 


106 


77 


1500 


429 


234 


151 


106 


77 


1600 


437 


236 


152 


107 


78 


1700 


444 


238 


153 


107 


78 


1800 


451 


240 


154 


107 


78 


1900 


457 


241 


155 


108 


78 


2000 


462 


243 


155 


108 


78 


2500 


485 


249 


158 


109 


79 


3000 


501 


253 


159 


no 


80 


3500 


513 


256 


161 


111 


80 


4000 


523 


25 9 


161 


in 


80 


4500 


530 


260 


162 


111 


80 


5000 


537 


262 


163 


112 


80 


6000 


546 


264 


164 


112 


81 


7000 


554 


266 


164 


n2 


81 


8000 


559 


267 


165 


113 


81 


9000 


563 


268 


16^ 


113 


81 


.0000 


567 


269 


165 


113 


81 
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SHORT TABL2 OF RANDOM NUMBERS 



44 


19 


15 


32 


63 


55 


34 


39 


80 


62 


24 


33 


74 


97 


80 


30 


65 


07 


22 


14 


61 


60 


86 


38 


40 


03 


96 


40 


03 


47 


52 


33 


76 


44 


56 


15 


37 


59 


20 


40 


93 


17 


11 


02 


55 


57 


48 


84 


10 


33 


79 


26 


34 


54 


67 


59 


28 


25 


47 


89 


98 


50 


75 


20 


09 


18 


24 


43 


23 


72 


80 


64 


39 


91 


63 


18 


38 


27 


74 


62 


19 


67 


54 


18 


91 


03 


35 


60 


81 


16 


42 


57 


66 


76 


72 


91 


P6 - 


36 


63 


06 


15 


03 


92 


70 


96 


70 


89 


80 


91 


08 


88 


53 


52 


13 


68 


85 


97 


74 


47 


53 


59 


54 


13 


09 


13 


80 


39 


18 


32 


69 


• 33 


46 


67 


43 


31 


09 


12 


60 


61 


75 


37 


19 


56 


90 


78 


10 


91 


11 


00 


63 


93 


23 


71 


58 


09 


78 


27 


55 


48 


82 


63 


89 


62 


13 


U 


71 


17 


23 


29 


89 


97 


47 


03 


13 


16 


94 


85 


82 


89 


07 


04 


32 


10 


59 


75 


12 


95 


71 


43 


68 


97 


18 


86 


05 


39 


14 


35 


48 


59 


30 


60 


10 


41 


31 


05 . 


45 


35 


40 


54 


03 


71 


85 


17 


74 


66 


27 


80 


20 


32 


80 


98 


00 


13 


50 


78 


02 


73 


39 


67 


92 


65 


41 


45 


36 


72 


56 


73 


44 


26 


04 


28 


86 


85 


64 


94 


11 


69 


57 


40 


80 


44 


94 


71 


20 


03 


30 


79 


25 


89 


98 


55 


98 


22 


45 


58 


74 


82 


81 


14 


02 


50 


54 


73 


81 


91 


07 


49 


33 


72 


90 


10 


20 


11 


85 


01 


43 


65 


02 


34 


22 


46 


41 


84 


74 


42 


64 


64 


58 


22 


75 


84 


05 


72 


90 


44 


27 


23 


39 


94 


00 


SO 


55 


04 


90 


51 


27 


61 


34 



77 


33 


29 


45 


00 


31 


67 


28 


11 


34 


79 


26 


30 


01 


84 


47 


45 


89 


71 


13 


33 


72 


08 


16 


60 


09 


21 


21 


18 


00 


75 


78 


73 


78 


19 


87 


24 


19 


90 


80 


87 


32 


36 


22 


67 


19 


20 


15 


33 


89 


74 


68 


48 


23 


65 


65 


20 


42 


23 


96 


34 


68 


02 


54 


87 


23 


27 


23 


46 


15 


36 


10 


78 


88 


84 


42 


32 


00 


92 


33 


69 


98 


96 


74 


97 


25 


14 


78 


21 


22 


63 


48 


46 


44 


01 


33 


38 


01 


58 


25 


37 


66 


14 


25 


49 


25 


94 


62 


82 


23 


00 


26 


36 


47 


05 


90 


14 


87 


48 


25 


29 


63 


03 


24 


64 


12 


19 


34 


03 


59 


28 


97 


57 


63 


78 


11 


80 


10 


39 


03 


56 


49 


92 


72 


63 


74 


58 


69 


03 


51 


03 


07 


71 


79 


32 


25 


59 


14 


72 


19 


17 


22 


25 


13 


85 


33 


35 


07 


86 


22 


45 


59 


98 


64 


30 


29 


89 


89 


80 


98 


84 


60 


93 


68 


16 


87 


17 


13 


08 


00 


50 


77 


18 


36 


57 


09 


62 


40 


69 


63 


77 


01 


89 


94 


96 


76 


27 


77 


94 


80 


19 


55 


56 


51 


36 


* 48 


92 


57 


51 


52 


83 


14 


82 


01 


28 


67 


51 


75 


96 


46 


21 


14 


39 


56 


81 


15 


35 


79 


26 


99 


78 


45 


36 


34 


45 


91 


82 


94 


93 


98 


01 


48 


17 


78 


.34 


54 


45 


04 


49 


82 


71 


57 


33 


28 


05 


77 


94 


65 


57 


70 


26 


25 


45 


49 


61 


22 


28 


44 


63 


95 


86 


75 


69 


56 


88 


34 


29 


64 


02 


57 


77 


47 


93 


72 


74 


91 


48 


46 


18 


34 


22 


07 


62 


17 


35 


34 


63 


27 


91 


70 


74 


13 


87 


44 


13 


50 


56 


48 



87 

31 

71 

33 

24 

47 

82 

74 

71 

11 

S4 

34 

10 

28 

61 

03 

72 

87 

04 

90 

42 

S8 

19 

75 

19 

08 

92 

29 

20 

17 

98 

85 

68 

00 

98 

85 

40 

66 

77 

62 

58 

60 

74 

12 • 

01 

81 

65 

85 

27 

81 

78 

31 

63 



106 



SI 

73 

03 

23 

73 

73 

96 

98 

97 

72 

47 

80 

41 

48 

44 

60 

83 

30 

33 

13 

19 

90 

63 

94 

02 

33 

27 

88 

85 

29 

21 

36 

28 

10 

60 

78 

25 

59 

51 

10 

82 

92 

28 

67 

72 

47 

19 

52 

99 

00 

15 

31 

53 



40 

62 

17 

36 

05 

36 

92 

34 

35 

25 

26 

56 

39 

07 

15 

79 

17 

37 

23 

24 

51 

25 

64 

53 

56 

97 

08 

38 

20 

42 

99 

06 

57 

51 

69 

75 

09 

31 

•93 

34 

34 

31 

75 

78 

46 

19 

09 

40 

29 

71 

37 

99 

99 

47 



29 


48 


30 


93 


43 


66 


29 


OS 


36 




35 


73 


57 


68 


36 


33 


91 


06 


98 


47 


48 


02 


42 


05 


32 


55 


02 


74 


39 


84 


24 


49 


79 


75 


83 


42 


00 


92 


33 




13 


7 c 


54 


39 


23 


39 


07 


17 


49 


13 


81 


^5 


32 


25 


70 


11 


17 


41 


64 


20 


30 


39 


87 


64 


37 


92 


35 


05 


43 


36 


98 


29 


97 


93 


37 


08 


30 


63 


21 


59 


69 


76 


02 


62 


31 


6 2 


47 


60 


92 


00 


04 ' 


94 


50 


05 


75 


82 


70 


80 


35 


11 


68 


25 


08 


95 


31 


79 


11 


79 


34 


05 


26 


37 


80 


39 


19 


06 


41 


02 


00 


53 


62 


59 


55 


05 


02 


16 


13 


17 


54 


43 


36 


19 


29 


28 


76 


49 


74 


39 


50 


78 


26 


IS 


41 


75 


64 


69 


61 


06 


38 


44 


04 


ca 


84 


80 


76 


51 


52 


41 


59 


01 


11 


05 


45 


11 


43 


40 


31 


84 


59 


43 


28 


10 


01 


65 


62 


07 


05 


59 


61 


31 


02 


65 


47 


47 


70 


39 


74 


22 


65 


97 


15 


70 


04 


89 


81 


78 


54 


84 


42 


95 


27 


52 


87 


47 


12 


52 


54 


62 


43 


38 


60 


36 


53 


56 


77 


06 


69 


03 


89 


91 


61 


04 


40 


33 


12 


06 


78 


91 


97 


88 


95 


20 


03 


64 


96 


60 


48 


01 


95 


44 


84 


69 


57 


92 


57 


U 


84 


44 


01 


33 


66 


53 


89 


81 


55 


87 


73 


81. 


58 


56 


42 


36 


25 


36 


49 


14 


34 


03 


52 


09 


20 


60 


11 


50 


46 


45 


86 


72 


50 


46 


11 


50 


46 


92 


45 


25 


48 


22 


23 


08 


32 


28 


87 


08 


74 


79 


91 


12 


43 


32 


03 


60 


19 


02 


70 


88 


72 


33 


20 


60 


86 


08 


64 


SO 


44 


34 


34 


24 


S3 


77 


32 


92 


32 


44 


40 


47 


10 


38 


22 


52 


96 


55 


31 


99 


73 


23 


40 


07 


64 


54 


44 


66 


33 


97 


47 


58 


42 


44 


88 


09 


28 


58 


70 


15 


74 


43 


62 


69 


82 


30 


77 


23 


77 


22 


25 


94 


80 


62 


95 


48 


98 


23 


86 


38 


66 


36 


87 


81 


16 


77 


30 


19 


36 


SO 


57 


77 


69 


60 


74 


22 


05 


77 


17 


77 


42 


59 


64 


99 


37 


03 


18 


03 


36 


69 


50 


59 


15 


79 


39 


42 


84 


18 


70 


39 


42 


48 


5.6 


84 


18 


70 


41 


74 


60 


88 


41 


20 


00 


15 


59 


60 


65 


65 


63 


78 


69 


24 


41 


65 


86 


10 


32 


00 


93 


35 


48 


15 


70 


11 


77 


83 


Cl 


91 


04 


02 


95 


63 


75 


74 


69 


69 


Cl 


34 


13- 


05 


' 57 


23 


06 


26 


23 


08 


66 


16 


23^ 


81 


37 


78 


16 


05 


57 


12 


46 


22 


90 


97 


39 


23 


71 


15 


08 


82 


64 


87 


29 


01 


20 


05 


42 


67 


98 


41 


67 


44 


28 


71 


45 


08 


76 


05 


83 


03 


34 


32 


62 


83 


27 


48 


33 


84 


60 


46 


18 


41 


23 


74 


73 


51 


72 


90 


95 


32 


80 


64 


75 


91 


98 


09 


40 


64 


39 


46 


79 


86 


53 


77 


78 


06 


62 


37 


48 


82 


78 


45 


13 


23 


32 


01 


09 


46 


36 


43 


66 


35 


20 


60 


97 


48 


21 


41 


84 


22 


72 


77 


83 


67 


91 


44 


83 


43 


25 


56 


33 


23 


80 


27 


86 


50 


76 


93 


96 


35 


68 


45 


37 


33 
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APPENDIX C 



SAMPLING TABLES USED IN THE FIELD TEST 

This appendix contains tables reproduced from those 
utilized at each activity. The first group of tables were 
used by PWC San Francisco at the CPL = 90% level. Great 
Lakes used the second group of tables at the CPL = 97% 
level . Both groups of tables were generated from the same 
formula. The tables provide graphic illustration of the 
effect that the CPL, surveillance level and population have 
on the sample size and CND. 
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C2ID/SS lABLS 



CRITICAL UUMBEE OF DEFECTIVZS (CHD)/SAMPLE SIZE (SS) 



Reduced Surveillancg f?age *1 of 2 ) 



> 




> 




















> 


> 


U1^ITS 


> 






CRITICAL : 


PSRPOHMAjrCS 


LEVEL 


XC7L) 






> 


> 


OF WORX 


> 




> 




> 




> 




> 




> 


> 


POPULAXIOS > 


951 


> 


9(Tt 


> 


85^ 


> 


8Ct 


> 


75X 


> 


> 


SIZE 


> 




> 




> 




> 




> 




> 


> 




> 


CTD/SS 


> 


CSD/SS 


> 


C!H)/SS 


> 


C2rt)/ss 


> 


cro/ss 


> 


> 


50 


> 


3/36 


> 


5/27 


> 


6/21 


> 


6/17 


> 


6/14 


> 


> 


75 


> 


4/47 


> 


6/33 


> 


6/25 


> 


7/19 


> 


7/26 


> 


> 


100 


> 


5/56 


> 


6/37 


> 


7/27 


> 


7/21 


> 


7/16 


> 


> 


125 


> 


6/62 


> 


7/40 


> 


7/29 


> 


7/22 


> 


7/17 


> 


> 


150 


> 


6/68 


> 


7/42 


> 


8/30 


> 


7/22 


> 


7/17 


•> 


> 


175 


> 


6/73 


> 


8/44 


> 


8/31 


> 


8/23 


> 


8/18 


> 


> 


200 . 


> 


7/77 


> 


6/46 


> 


8/31 


> 


8/23 


> 


8/18 


> 


> 


225 


> 


7/80 


> 


8/47 


> 


8/32 


> 


8/24 


> 


8/18 


> 


> 


Z50 


> 


7/83 


> 


8/48 


> 


8/32 


> 


8/24. 


> 


8/18 


> 


> 


275 


> 


7/86 


> 


8/49 


> 


8/33 


> 


8/24 


> 


8/18 


> 


> 


300 


> 


8/88 


> 


8/49 


> 


8/33 


> 


8/24 


> 


8/19 


> 


> 


325 


> 


8/90 


> 


9/50 


> 


8/33 


> 


8/24 


> 


8/19 


> 


> 


350 


> 


8/92 


> 


9/51 


> 


9/34 


> 


8/24 


> 


8/19 


> 


> 


375 


> 


8/94 


> 


9/51 


> 


9/34 


> 


8/25 


> 


8/19 


> 


> 


400 


> 


8/95 


> 


9/52 


> 


9/34 


> 


8/25 


> 


8/19 


> 


> 


425 


> 


8/97 


> 


9/52 


> 


9/34 


> 


8/25 


> 


8/19 


> 


> 


450 


> 


8/93 


> 


9/52 


> 


9/34 


> 


8/25 


> 


8/19 


> 


> 


475 


> 


9/99 


> 


9/53 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


500 


> 


9/100 


> 


9/53 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


550 


> • 


9/102 


> 


9/53 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


600 


> 


9/103 


> 


9/54 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


650 


> 


9/105 


> 


9/54 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


700 


> 


9/106 


> 


9/55 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


750 


> 


9/107 


> 


9/55 


> 


9/35 


> 


8/25 


> 


8/19 


> 


> 


800 


> . 


9/108 


> 


9/55 


> 


9/36 


> 


8/25 


> 


8/19 


> 


> 


850 


> 


9/109 


> 


9/55 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


900 


> 


9/110 


> 


10/56 


> 


9/36 


> 


9/26 


> 


8/19 




> 


950 


> 


9/110 


> 


10/56 


> 


9/36 


> 


9/26 


> 


8/19 




> 


1000 


> 


10/111 


> 


10/56 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1100 


> 


10/112 


> 


10/56 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1200 


> 


10/113 


> 


10/56 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1300 


> 


10/114 


> 


10/57 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1400 


> 


10/115 


> 


10/57 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1500 


> 


10/115 


> 


10/57 


> 


9/36^ 


> 


9/26 


> 


8/19 


> 


> 


1600 


> 


10/116 


> 


10/57 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1700 


> 


10/116 


> 


10/57 


> 


9/36 


> 


9/26 


> 


8/19 


> 


> 


1800 


> 


10/U7 


> 


10/57 


> 


9/36 


> 


9/26 


> 


8/20 


> 


> 


1900 


> 


10/117 


> 


10/57 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


2000 


> 


10/118 


> 


10/57 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


2500 


> 


10/119 


> 


10/58 


> 


9/37 


> 


9/26 


> 


8/20 


> 
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CSO/SS TABLZ 



CRITICAL tfUMBER OF DEFECTIVES (Cin))/SAMPLE SIZE (SS) 



Rftduced Surveillance CPage 2 of 2 ) 



> 




> 




















> 


> 


uinrs 


> 






CRITICAL PERFORMANCE 


LEVEL 


rcPL) 






> 


> 


OF WORX 


> 




> 




> 




> 




> 




> 


> 


POPULAXIOtf > 


95X 


> 


9cn 


> 


851 


> 


8(n 


> 


751 


> 


> 


SIZE 


> 




> 




> 




> 




> 




> 


> 




> 


CSD/SS 


> 


CITD/SS 


> 


CIID/SS 


> 


OTD/SS 


> 


ClID/SS 


> 


> 


3000 


> 


10/120 


> 


10/58 


> 


9/37 


> 


9/26 


> 


6/20 


> 


> 


3500 


> 


10/121 


> 


10/58 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


4000 


> 


10/121 


> 


10/58 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


4500 


> 


10/122 


> 


10/58 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


5000 


> 


10/122 


> 


10/58 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


6000 


> 


10/122 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


7000 


> 


11/123 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


8000 


> 


11/123 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


9000 


> 


11/123 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


10000 


> 


11/123 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


20000 


> 


11/124 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


30000 


> 


11/124 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


40000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


50000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


60000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


* 


70000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


80000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


90000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


100000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


250000 


> 


11/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> 


> 


500000 


> 


U/125 


> 


10/59 


> 


9/37 


> 


9/26 


> 


8/20 


> • 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> ' 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 



y 
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CSD/SS TiBLg 



CRITICAL tfUKSER OF DEFECTIVES (CUD) /SAMPLE SIZE CSS) 



Mormal Surveillance (Page 1 of 2 ) 



> 




> 




















> 


> 


uiriTS 


> 






CRITICAL PERFORMAHCE 


LEVEL 








> 


> 


OF WORK 


> 




> 




> 




> 




> 




> 


> 


POPtrULTIOM 




95X 


> 


90X 


> 


85X 


> 


80X 


> 


75X 


> 


> 


SIZE 


> 




> 




> 




> 




> 




> 


> 




> 


cro/ss 


> 


CND/SS 


> 


CSD/SS 


> 


cro/ss 


> 


CTTO/SS 


> 


> 


50 


> 


4/40 


> 


6/33 


> 


7/28 


> 


8/23 


> 


8/20 


> 


> 


75 


> 


5/55 


> 


7/42 


> 


9/34 


> 


9/27 


> 


9/23 


> 


> 


100 


> 


6/67 


> 


8/49 


> 


10/38 


> 


10/30 


> 


10/25 


> 


> 


125 


> 


7/78 


> 


9/55 


> 


10/41 


> 


11/32 


> 


11/26 


> 


> 


150 


> 


8/87 


> 


10/59 


> 


11/44 


> 


11/34 


> 


11/27 


> 


> 


175 


> 


8/95 


> 


11/63L 


> 


11/45 


> 


12/35 


> 


11/27 


> 


> 


200 . 


> 


9/101 


> 


11/66 


> 


12/47 


* > 


12/36 


> 


U/28 


> 


> 


225 


> 


9/107 


> 


_ 11/68.- 


> 


12/48 


> 


12/36 


> 


. U/28 


> 


> 


250 


> 


10/113 


> 


U/70 


> 


12/49 


> 


12/37 


> 


12/49 


> 


> 


275 


> 


10/118 


> 


12/72 


> 


12/50 


> 


12/37 


> 


12/29 


> 


> 


300 


> 


10/122 


> 


12/74 


> 


13/51 


> 


U/38 


> 


12/29 


> 


> 


325 


> 


11/126 


> 


13/75 


> 


13/52 


> 


13/38 


> 


12/30 


> 


> 


350 


> 


11/130 


> 


13/76 


>- - 


13/52 


> 


U/39 


> 


U/30 


> 


> 


375 


> 


11/133 


> 


13/77 


> 


13/53 


> 


13/39 


> 


12/30 


> 


> 


400 


> 


12/136 


> 


13/78 


> 


U/53 


> 


13/39 


> 


12/30 


> 


> 


425 


> 


12/139 


> 


13/79 


> 


13/54 


> 


13/39 


> 


12/30 


> 


> 


450 


> 


12/141 


> 


13/80 


> 


U/54 


> 


13/39 


> 


U/30 


> 


> 


475 


> 


12/144 


> 


14/81 


> 


13/54 


> 


13/40 


> 


12/30 


> 


> 


500 


> 


12/146 


> 


14/82 


> 


14/55 


> 


U/40 


> 


12/30 


> 


> 


550 


> 


13/150 


> 


14/83 


> 


14/55 


> 


13/40 


> 


13/31 


> 


> 


600 


> 


U/153 


> 


. 14/84 


> 


14/56 


> 


13/40 


> 


U/31 


> 


> 


650 


> 


13/156 


> 


14/85 < 


> 


14/56 


> 


13/41 


> 


13/31 


> 


> 


700 


> 


13/159 


> 


14/86 


> 


14/56 


> 


U/41 


> 


U/31 


> 


> 


750 


> 


14/161 


> 


14/85 


> 


14/57 


> 


13/41 


> 


13/31 


> 


> 


800 


> 


14/164 


> 


15/87 


> 


14/57 


> 


U/41 


> 


U/31 


> 


> 


850 


> 


14/166 


> 


15/87 


> 


14/57 


> 


13/41 


> 


13/31 


> 


> 


900 


> 


14/167 


> 


15/88 


> 


14/57 


> 


U/41 


> 


U/31 


> 


> 


950 


> 


14/169 


> 


15/88 


> 


14/58 


> 


13/41 


> 


13/31 


> 


> 


^1000 


> 


14/172 


> ' 


15/89 


> 


14/58 


> 


u/41 


> 


U/31 


> 


> 


1100 


> 


15/173 


> 


15/89 


> 


14/58 


> 


14/42 


> 


13/32 


> 


> 


1200 


> 


15/176 


> 


15/90 


> 


14/58 


> 


14/42 


> 


U/32 


> 


> 


1300 


> 


15/178 


> 


15/91 


> 


15/59 


•> 


14/42 


> 


13/32 


> 


> 


1400 


> 


15/179 


> 


15/91 


> 


15/59 


> 


14/42 


> 


U/32 


> 


> 


1500 


> 


15/181 


> 


_^5/91 


> 


15/59 


> 


14/42 


> 


13/32 


> 


> 


1600 


> 


15/182 


> 


15/92 


> 


15/59 


> 


14/42 


> 


U/32 


> 


> 


17(70 


> 


15/183 


> 


15/92 


> 


15/59 


> 


14/42 


> 


U/32 


> 


> 


1800 


> 


16/185 


> 


15/92 


> 


15/59 


> 


14/42 


> 


U/32 


> 


> 


1900 


> 


16/186 


> 


15/93 


> 


15/59 


> 


14/42 


> 


13/32 


> 


> 


2000 


> 


16/186 


> 


15/93 


> 


15/60 


> 


14/42 


> 


u/32 


> 


> 


2500 


> 


15/190 


> 


16/94 


> 


15/60 


> 


14/43 


> 


13/32 


> 
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CMD/SS TABLE 



CRITICAL NUMBER OF DEFECTIVES (CND) /SAMPLE SIZE (SS) 



Normal Surveillance (Page 2 of 2 ) 



> 




> 




















> 


> 


UNITS 


> 






CRITICAL PERFORMANCE 


LEVEL 


(CPL) 






> 


> 


OF WORK 


> 




> 




> 




> 




> 




> 


> 


POPULATION 


> 


951 


> 


90X 


> 


85X 


> 


SOX 


> 


75X 


> 


> 


SIZE 


> 




> 




> 




> 




> 




> 


> 




> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


> 


3000 


> 


16/192 


> 


16/94 


> 


15/60 


> 


14/43 


> 


13/32 


> 


> 


3500 


> 


16/194 


> 


16/95 


> 


15/60 


> 


14/43 


> 


13/32 


> 


> 


4000 


> 


16/195 


> 


16/95 


> 


15/60 


> 


14/43 


> 


13/32 


> 


> 


4500 


> 


17/197 


> 


16/95 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


5000 


> 


17/198 


> 


16/96 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


6000 


> 


17/199 


> 


16/96 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


7000 


> 


17/200 


> 


16/96 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


8000 


> 


17/201 


> 


16/96 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


9000 


> 


17/201 


> 


16/96 


> 


15/61 


. > 


14/43 


> 


13/32 


> 


> 


10000 


> 


17/202 


> 


16/96 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


20000 


> 


17/204 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


30000 


> 


17/204 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


40000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


50000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


60000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


70000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


80000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


90000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


100000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


250000 


> 


17/205 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 


500000 


> 


17/206 


> 


16/97 


> 


15/61 


> 


14/43 


> 


13/32 


> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 
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> 
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> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




*> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 
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CND/SS TABLE 



CRITICAL NUMBER OF DEFECTIVES (CND) /SAMPLE SIZE (SS) 



> 
> 
> 
> 

> SIZE >>>>>> 



> 




> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


> 


50 


> 


4/45 


> 


7/40 


> 


9/36 


> 


11/32 


> 


11/28 


> 


> 


75 


> 


6/63 


> 


9/54 


> 


12/47 


> 


13/40 


> 


14/35 


> 


> 


100 


> 


7/80 


> 


11/66 


> 


24/55 


> 


15/46 


> 


16/39 


> 


> 


125 


> 


8/95 


> 


13/76 


> 


15/62 


> 


17/51 


> 


17/43 


> 


> 


•150 


> 


9/110 


> 


14/85 • 


> 


17/68 


> 


18/55 


> 


18/43 


> 


> 


175 


> 


10/123 


> 


15/92 


> 


18/72 


> 


19/56 


> 


19/47 


> 


> 


200 


> 


11/135 


> 


16/99 


> 


19/76 


> 


20/60 


> 


20/49 


> 


> 


225 


> 


12/145 


> 


17/104 


> 


19/79 


> 


20/63 


> 


20/50 


> 


> 


250 


> 


13/156 


> 


18/110 


> 


20/82 


> 


21/64 


> 


21/52 


> 


> 


275 


> 


14/165 


> 


19/114 


> 


21/85 


> 


21/65 


> 


21/53 


> 


> 


300 


> 


15/174 


> 


20/118 


> 


21/87 


> . 


22/67 


> 


21/53 


> 


> 


325 


> 


15/182 


> 


20/122 


> 


22/89 


> 


22/68 


> 


22/54 


> 


> 


350 


> 


16/189 


> 


21/125 


> 


22/91 


> 


22/69 


> 


22/55 


> 




. 375 


> 


16/196 


> 


21/128 


> 


23/92 


> 


23/70 


> 


22/55 


> 




400 


> 


17/203 


> 


22/131 


> 


23/94 


> 


23/70 


> 


22/56 


> 


> 


425 


> 


18/209 


> 


22/134 


> 


23/95 


> 


23/72 


> 


22/56 


> 


> 


450 


> 


18/215 


> 


22/136 


> 


24/96 


. > 


24/73 


> 


23/57 


> 


> 


475 


> 


18/220 


> 


23/133 


> 


24/98 


> 


24/73 


> 


23/57 


> 


> 


500 


> 


19/226 


> 


23/140 


> 


24/99 


> 


24/74 


> 


23/58 


> 


> 


550 


> 


20/235 


> 


24/144 


> 


24/100 


> 


24/75 


> 


23/58 


> 


> 


600 


> 


20/244 


> 


24/147 


> 


25/102 


> 


25/76 


> 


24/59 


> 


> 


650 


> 


21/252 


> 


25/150 


> 


25/103 


> 


25/76 


> 


24/59 


> 


> 


700 


> 


22/259 


> 


25/152 


> 


25/104 


> 


25/77 


> 


24/59 


> 


> 


750 


> 


22/266 


> 


25/155 


> 


26/105 


> 


25/78 


> 


24/60 


> 


> 


800 


> ’ 


23/272 


> 


26/157 


> 


26/106 


> 


25/78 


> 


24/60 


> 


> 


850 


> 


23/277 


> 


26/159 


> 


26/107 


> 


25/79 


> 


24/60 


> 


> 


900 


> 


23/282 


> 


26/159 


> 


26/107 


> 


25/79 


> 


24/60 


> 


> 


950 


> 


24/287 


> 


27/162 


> 


27/109 


> 


26/79 


> 


24/61 


> 


> 


1000 


> 


24/292 


> 


27/163 


> 


27/109 


> 


26/80 


> 


24/61 . 


> 


> 


1100 


> 


2S/299 


> 


27/166 


> 


27/110 


> 


26/80 


> 


24/61 


> 


> 


1200 


> 


25/306 


> 


28/168 


> 


27/111 


> 


26/81 


> 


25/62 


> 


> 


1300 


> 


26/313 


> 


28/170 


> 


27/112 


> 


26/81 


> 


25/62 


> 


> 


1400 


> 


26/318 


> 


28/171 


> 


28/113 


> 


26/82 


> 


25/62 


> 


> 


1500 


> 


27/323 


> 


28/173 


> 


28/113 


> 


26/82 


> 


25/62 


> 


> 


1600 


> 


27/327 


> 


29/174 


> 


28/114 


> 


26/82 


> 


25/62 


> 


> 


1700 


> 


27/331 


> 


29/175 


> 


28/114 


> 


26/82 


> 


25/63 


> 


> 


laoo 


> 


28/335 


> 


29/176 


> 


28/115 


> 


27/83 


> 


25/63 


> 


> 


1900 


> 


28/335 


> 


29/177 


> 


28/115 


> 


27/83 


> 


* 25/63 


> 




2000 


> 


28/341 


> 


29/178 


> 


28/116 


> 


27/83 


> 


25/63 


> 




2500 


> 


29/353 


> 


30/181 


> 


29/117 


> 


27/84 


> 


25/63 


> 



Tightened Surveillance (Page 1 of 2 ) 

> > 

> UNITS > CRITICAL PERFORMANCE LEVEL CCPL) 

> OF WORX > > > > > 

> POPULATION > 95X > 90% > ‘ 85% > 80X > 75% 
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aro/SS TABLE 



CRITICAL NUMBER OF DEFECTIVES (CND) /SAMPLE SIZE (SS) 



Tightened Surveillance (Page 2 of 2 ) 



> 




> 




















> 


> 


UNITS 


> 






CRITICAL PERFORMANCE 


LEVEL 


_.CCPL) 






> 


> 


OF WORK 


> 




> 




> 




> 




> 




> 


> 


POPULATION 


> 


95X 


> 


90% 


> 


85% 


> 


80% 


> 


75% 


> 


> 


SIZE 


> 




> 




> 




> 




> 




> 


> 




> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


CND/SS 


> 


> 


3000 


> 


30/362 


> 


30/183 


> 


29/118 


> 


27/84 


> 


25/64 


> 


> 


3500 


> 


30/368 


> 


30/185 


> 


29/119 


> 


27/85 


> 


25/64 


> 


> 


4000 


> 


31/373 


> 


30/186 


> 


29/119 


> 


27/85 


> 


26/64 


> 


> 


4500 


> 


31/377 


> 


31/187 


> 


29/119 


> 


27/85 


> 


26/64 


> 


> 


5000 


> 


31/380 


> 


31/188 


> 


29/120 


> 


27/85 


> 


26/64 


> 


> 


6000 


> 


31/385 


> 


31/189 


> 


29/120 


> 


27/85 


> 


26/64 


> 


> 


7000 


> 


31/389 


> 


31/190 


> 


29/121 


> 


28/86 


> 


26/64 


> 


> 


8000 


> 


31/391 


> 


31/190 


> 


29/121 * 


> 


28/86 


> 


26/64 


> 


> 


9000 


> 


31/394 


> 


31/191 


> 


30/121 


> 


28/86 


> 


26/65 


> 


> 


10000 


> 


31/395 


> 


31/191 


> 


30/121 


> 


28/86 


> 


26/65 


> 


> 


20000 


> 


31/403 


> 


31/193 


> 


30/122 


> 


28/86 


> 


26/65 


> 


> 


30000 


> 


31/406 


> 


31/194 


> 


30/122 


> 


28/86 


> 


26/65 


> 


> 


40000 


> 


31/407 


> 


31/194 


> 


30/122 


> 


28/86 


> 


26/65 


> 


> 


50000 


> 


31/408 


> 


31/194 


> 


30/122 


> 


28/86 


> 


26/65 


> 


> 


60000 


> 


31/409 


> 


31/194 


> 


30/122 


> 


28/87 


> 


26/65 


> 


> 


70000 


> 


31/409 


> 


31/194 


> 


30/123 


> 


28/87 


> 


26/65 


> 


> 


80000 


> 


31/409 


> 


31/194 


> 


30/123 


> 


28/87 


> 


26/65 


> 


> 


90000 


> 


31/410 


> 


31/195 


> 


30/123 


> 


28/87 


> 


26/65 


> 


> 


100000 


> 


31/410 


> 


31/195 


> 


30/123 


> 


28/87 


> 


26/65 


> 


> 


250000 


> 


31/411 


> 


31/195 


> 


30/123 


> 


28/87 


> 


26/65 


> 


> 


500000 


> 


31/4U . 


> 


31/195 


> 


30/123 


> 


28/87 


> 


26/65 


> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 
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> 
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> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




>* 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 


> 




> 




> 




> 




> 




> 




> 
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CND/SS TABLE 



REDUCED SURVEILLANCE 



N 


997. 


987. 


97X 96X 9SX 


90 


1 / 44 


2/43 


3/40 3 / 


30 3 / 


34 


71 


2/47 


3/41 


3/99 4 / 


91 4 / 


47 


too 


2 / »7 


3/74 


4/40 4 


/ 41 9 / 


94 


tai 


3 / 109 


3/40 


4/74 9 


/ 70 4 


/ 43 


190 


3/132 


4/102 


9/00 


4/77 4 


/ *• 


179 


3 / 130 


4 / 113 


9/44 


4 / B3. 4 


/ 73 


700 


3/193 


9/133 


4 / 103 


4 / H 


7/77 


239 


3 / 147 


9/132 


4 / 104 


7/43 


7 / iO 


290 


3 / 1«1 


9 / 141 


4 / 119 


7/47 


7 / W3 


379 


4 / IT3 


9 / Mi 


4 / 130 


7/100 


7/04 


300 


4 / 309 


4 / 199 


7 / 134 


7/103 


i / M 


339* 


4 / 217 


4/142 


7/134 


7/104 


i / 40 


390 


4 / 23S 


4/140 


7/133 


0 / 104 


i / 42 


3T9 


4 / 330 


4/173 


7/134 


i / 111 


i / 44 


400 


9 / 240 


4 / I7i 


7/134 


• / 113 


i / 49 


429 


9 / 397 


4/103 


7 / m3 


i / 119 


i / 47 


490 


9 / 344 


7 / 100 


• / M4 


■ /*117 


i / 4i 


479 


9/379 


7/142 


■ / 147 


■ / Hi 


4/44 


900 


9 / 3«3 


7 / 144 


■ / 144 


i / 130 


4/100 


990 


9 / 240 


7/303 


i / 193 


4/133 


4 / 103 


400 


4/313 


7 / 310 


• / 197 


4 / 139 . 


4 / 103 


490 


4/339 


■ / 319 


• / 140 


4/137 


4 / 109 


700 


4/307 


• / 231 


• / 143 


4/134 


4/104 


790 


4 / 340 


■ / 229 


4 / 144 


4/130 


4 / 107 


•OO 


4 / 394 


• / 330 


4/1*0 


4/132 


4 / lOi 


■90 


7 / 344 


• / 334 


4 / 170 


4 / 133 


4 / 104 


400 


7 / 37« 


• / 207 


4/172 


4/134 


4 / 110 


♦90 


7 / 3B4 


• / 34t 


4 / 174 


4/139 


4 / 110 


lOOO 


7 / 344 


• / 244 


4/179 


4/134 


10 / HI 


1100 


7 / 404 


4 / 244 


4/170 


4/130 


10 / 113 


1300 


7 / 422 


4/394 


4 / lil 


10 / 134 


10 / 113 


1300 


• / 434 


4/390 


4 / 103 


10 / Ml 


10 / 114 


MOO 


• / 444 


4 / 343 


10 / 109 


10 / M3 


10 / 119 


1900 


• / 494 


4 / 349 


10 / 104 


10 / M3 


10 / 119 


1400 


• / 443 


4 / 340 


10 / IM 


10 / 144 


10 / 114 


1700 


• / 471 


4/371 


10 / 1»4 


10 / 144 


10 / 114 


i«oo 


• / 470 


4 / 373 


10 / 140 


10 / 149 


10 / 117 


1400 


• / 409 


4/379 


10 / 141 


10 / 144 


10 / 117 


3000 


4 / 441 


10/277 


10 / 143 


10 / 144 


10 / Hi 


3900 


4 / 914 


10 / 2*9 


10 / 144 


10 / Mi 


10 / 114 


3000 


• 4 / 939 


10 / 341 


10 / 140 


10 / 190 


10 / 130 


3900 


4 / 944 


10/349 


10 / 200 


10 / I9t 


10 / 181 


4000 


10/940 


10/340 10/203 


10 / 133 10 / 131 
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CND/SS TABLE 



NORMAL SURVE I LLANCE 
N 99% 98% 97% 96% 93% 

90 l/4« 2/4« 3/44' 3/ 43 4/40 

79 3/70 3/44 4/43 4/90 9/ 99 



too 


3 / 


71 


3 / »4 4 


/ 70 9 


/ 72 * / 


47 


i39 


3 / 


113 


4 / 101 


9/73 


4/04 7 


/ 70 


190 


3 / 


133 


4 / 117 


4/109 


7/79 0 


/ 97 


179 


3 / 


190 


9/133 


4 / 117 


7 / 109 


• / 79 


300 


3 / 


147 


9 / 149 


7/137 


■ / 113 


7 / 101 


339 


4 / 


1*4 


4 / 19« 


7 / 137 


0/131 


7 / 107 


390 


4 / 


303 


4/170 


■ / 14* 


7 / 127 


10 / 113 


379 


4 / 


317 


4 / 101 


0/194 


7/134 


10 / SSI 


W 


4 / 


234 


7/172 


■ / 143 


10 / 137 


10 / 133 


339 


9 / 


347 


7 / 303 


7 / 147 


to / 144 


IS / 134 


390 


9 / 


344 


7 / 311 


7 / 179 


10 / 147 


11 / 130 


379 


9 / 


Z 79 


0/330 


7 / til 


to / 193 


il / 133 


400 


9 / 


371 


• / 22 a 


10 / i97 


11 / 997 


S3 / 134 


439 


9 / 


304 


■ / 334 


to / 173 


It / 141 


13 / 137 


490 


4 / 


317 


• / 343 


to / 177 


IS / 149 


13 / 141 


479 


4 / 


337 


7/391 


10/303 


11 / 140 


13 / 144 


900 


4 / 


341 


7 / 397 


IL / 304 


13 / 171 


S3 / 144 


990 


4 / 


343 


7/270 


11 / 314 


13 / 174 


13 / 190 


400 


7 / 


3B9 


10 / 7S3 


11 / 231 


13 / IIS 


13 / 193 


490 


7 / 


409 


10 / 373 


13 / 23^ 


13 / 104 


13 / 194 


700 


7 / 


433 


10 / 303 


13 / 


S3 / 1*7 


13 / 197 


790 


■ / 


441 


IS / 311 


13/237 


13 / 173 


14 / 141 


•00 


• / 


49« 


11 / 317 


13 / 343 


S3 / 174 


14 / 144 


•90 


■ / 


474 


11/337 


13 / 340 


13 / 177 


14 / 14* 


TOO 


7 / 


407 


11/334 


13/393 


14 / 303 


14 / 147 


790 


7 / 


904 


13 / 340 


13 / 394 


14 / 304 


14 / 147 


1000 


7 / 917 


13 / 347 


13 / 397 


14 / 304 


14 / 171 


1100 


7 i 


' 943 


13 / 390 


14 / 3*^ 


14 / 310 


19 / 173 


1300 


10 


/ 944 


13 / 340 


14 / 371 


14 / 314 


19 / 174 


1300 


10 


/ 9*7 


13 / 377 


14 / 374 


19 / 317 


19 / 17* 


1400 


10 


/ 407 


13 / 309 


14 / 300 


19 / 317 


19 / 177 


1900 


1 1 


/ 439 


13 / 372 


14 / 244 


19 / 331 


19 / tot 


1400 


11 


/ 443 


14 / 37* 


19 / 997 


19 / 233 


19 /> t«a 


1700 


11 


/ 497 


14 / 404 


19 / 370 


19 / 339 


19 / 103 


i»oo 


13 


/ 473 


14 / 410 


19/373 


19/337 


14 / 109 


1700 


13 


/ 409 


14 / 419 


19/37* 


19 / 237 


14 / 104 


3000 


13 


/ 47 « 


14 / 417 


19 / 37« 


19 / 330 


14 / 104 


3900 


13 


/ 790 


19 / 430 


1* / 307 


14 / 339 


14 / 170 


3000 


13 


/ 770 


19 / 491 


1* / 313 


14/337 


14 / 172 


3900 


14 


/ 430 


14 / 441 


1* / 310 


14 / 343 


14 / 174 


4000 


14 


/ 049 


14 / 44« 


1* / 333 


14 / 344 


14 / 174 
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CND/SS TABLE 



TIGHT SURVEILLANCE 



N 99% 987. 97X 96X 937. 



90 


1 / 40 


3/40 3 


/ 47 3 / 


44 4 / 49 




79 

100 


3/73 

3/04 


3/70 4 

4/01 9 


/ 40 9 / 

/ 00 4 / 


44 4 / 43 

04 7/00 


m 


3 / lit 


4 / 113 


4 / 104 


7/101 0 


/ 04 


190 


3 / 140 


9 / 131 


7 / 134 


0/114 0 


/ ito 


179 


3/143 


9/190 


7 / 140 


0 / 131 10 / 133 


300 


3 / t«3 


4 / 140 


0 / 194 


10 / 144 


11 / 139 


339 


4 / 304 


7 / 104 


0 / 170 


11 ./ 197 


12 / 149 


390 


4 / 334 


7 / 303 


10 / 104 


13 / 140 


13 / 194 


379 


4 / 344 


0/310 


10 / 107 


13 / 100 


14 / 149 


300 


t / 343 


• / *37 


It / tio 


13 / too 


to / 174 


» 


9 / 3oa 


9 / 977 


11 / 233 


14 / 300 


ft / 108 


300 


t / 301 


7 / 043 


It / B3 


14 / 300 


44 / too 


379 


4 / 310 


10 / 377 


13 / 344 


19 / 310 


14 / 104 


400 


4/337 


10 / 301 


13 / 399 


19/234 


17 / 303 


•39 


4 / 399 


10 / 303 


13 / 244 


14 / 334 


10 / 300 


•90 


7/373 


11 / 314 


14 / 374 


14 / 341 


10 / 319 


• 79 


7 / 307 


11 / 330 


14 / 303 


17 / 340 


10 / 230 


900 , 


7 / 409 


13/340 


19 / 373 


17 / 399 


10 / 334 


990 


• / 430 


13/343 


14 / 300 


10 / 347 


30 / 339 


400 


9 / 444 


13/303 


14/323 


lO / 270 


30/344 


*»0 


7 / 400 


14 / 403 


17 / 337 


10 / 300 


31 / 393 


700 


7 / 930 


14 / 423 


10 / 390 


30 / 300 


23 / 390 


790 


10 / 994 


19 / 430 


lO / 343 


31 / 307 


23 / 344 


ooo 


10 / 9C3 


19 / 494 


17/373 


31 / 319 


23/372 


990 


10 / 400 


14 / 473 


lO / 304 


23/323 


23 / 277 


700 


11 / 434 


14 / 40T7 


30/304 


23/330 


23 / 303 


790 


11 / 490 


17 / 901 


30 / 403 


23/334 


34 / 307 


1000 


13 / 403 


17 / 919 


31 / 413 


S3 / 343 


24/303 


1100 


13 / 777 


lO / 940 


31 / 430 


34 / 393 


29 / 377 


1700 


13 / 740 


10/943 


23 / 443 


34 / 343 


39 / 304 


1300 


14 / 907 


30/904 


33 / 499 


39 / 371 


34 / 313 


1400 


14 / 047 


30 / 404 


33 / 447 


39/370 


34 / 310 


l»O0 


19/903 


31/433 


“34 / 477 


34 / 304 


27/333 


1400 


14 / 014 


31 / 430 


34 / 407 


34 / 303 


37 / 337 


1700 


14 / 040 


23 / 493 


39 / 404 


34 / 300 


27/331 


1*00 


17 / 070 


33/440 


39/304 


37/403 


30/339 


1700 


17 / 1007 


23/401 


34 / 913 


37 / 400 


20/330 


3000 


17 / 1039 


23 / 403 


34 / 910 


37 / 413 


30/341 


3900 


10 / 1194 


39 / 749 


37 / 947 


30 / 430 


30 / 393 


3000 


31 / 1390 


34 / 704 


30 / 97S 


30 / 443 


30/343 


3900 


33 / 1330 


37 / 014 


30 / 904 


30 / 493 


30/340 


4000 


33 / 1304 


SO / 030 


30 / 904 


30 / 440 


31/373 
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APPENDIX D 



RSED V3.0 TABLES AND PAYMENT ANALYSIS 



This appendix contains inspection tables and deduction 
adjustments (deltas) for RSED V3.0. That is followed by a 
sample payment analysis. The material within this appendix 
has been reproduced from preliminary NAVFAC guidance on the 
proposed method [Ref. 19]. Interesting features are the 
enhanced inspection requirements and narrowly defined 
population ranges. There is no CND because extrapolation 
takes place in the event of any defect. 
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R5ED V3.0 - Table o-f Sample Sizes 
For Normal Sampling Levels 

07-07-1986 

Sample sizes are "for the indicated monthly population: 



1 1 a t i on 


range - 


Sampl.e Size 


Popu 1 at i on 


range - 


Samp 1 e Size 


33 


_ 


34. , , 


30 


35 - 


36. . , 




37 


- 


37. . , 


32 


38 - 


38. . . 


33 


39 


- 


40. . . 




41 - 


41 . . . 


35 


42 


- 


43.., 


36 


44 - 


44. . . 




45 


- 


46. . . 


38 


47 - 


47. . , 




48 


- 


49. . , 


40 


50 - 


50. . , 


4 1 


51 


- 


52. . , 




53 - 


53. . , 


43 


54 


- 


55. . , 


44 


56 - 


56. . . 


45 


57 


- 


58. . , 


46 


59 - 


60. . . 


47 


61 


- 


61 . . , 




62 - 


63. . . 


49 


64 


- 


65. . . 




66 - 


66 . . . 


51 


67 


- 


68. . , 


52 


69 - 


70. . . 




71 


- 


71 . . , 


54- 


72 - 


73. . . 




74 


- 


75. . . 


56 


76 - 


77. . . 


57 


78 


- 


79. . , 


58 


80 - 


81 . . . 


59 


82 


- 


83. . . 




84 - 


84. . . 




85 


- 


86. . , 


62 


87 - 


88. . . 


63 


89 


- 


90. . . 




91 - 


92. . . 




93 


- 


95. . , 


66 


96 - 


97. . . 




98 


- 


99. . . 


68 


100 - 


101 .. . 




102 


- 


103. . . 


70 


104 - 


105. . . 


71 


106 


- 


107. . . 


72 


108 - 


1 10. . . 


73 


1 1 1 


- 


112... 




113 - 


114... 




115 


- 


117... 


76 


118 - 


119... 




120 


- 


122. . . 


78 


123 - 


124. . . 




125 


- 


127. . . 




128 - 


129. . . 


81 


130 


- 


132. . . 




133 - 


134. . . 


83 


135 


- 


137. . . 




138 - 


140. . . 




141 


- 


142. . . 


86 


143 - 


145. . . 




146 


- 


148. . . 


88 


149 - 


151 .. . 




152 


- 


154. . . 


90 


155 - 


157. . . 


91 


158 


- 


160. . . 


92 


161 - 


163. . . 


93 


164 


- 


166. . . 


94 


167 - 


169. . . 




170 


- 


172. . . 




173 - 


176. . . 


*97 


177 


- 


179. . . 




180 - 


182. . . 


99 


183 


- 


186. . . 


100 


187 - 


189. . . 


101 


190 


- 


193. . . 


102 


194 - 


196. . . 


103 


197 


- 


200. . . 


104 


201 - 


204. . . 




205 




208. . . 


106 


209 - 


211... 


^ 107 


212 


- 


215. . . 




216 - 


219. . . 


109 


220 


- 


224. . . 


110 


225 - 


228. . . 


Ill 


229 


- 


232. . . 


112 


233 - 


236. . . 


113 


237 


- 


241 .. . 


114 


242 - 


245. . . 


115 


246 


- 


250. . . 




251 - 


255. . . 


117 


256 


- 


259. . . 




260 - 


264. . . 




265 


- 


269. . . 


120 


270 - 


274. . . 




275 


- 


’ 280... 


122 


281 - 


285. . . 
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R5ED V3.0 - Table oi Sample Sizes 
For Normal Sampling Levels 



Sample sizes are for the indicat 



Population range - Sample Size 

2B6 - 290 124 

297 - 302 126 

309 - 313 12B 

321 - 326 130 

333 - 339 132 

346 - 352 134 

360 - 366 136 

375 - 3B1 . 13B 

390 - 397 140 

406 - 414 142 

423 - 431 144 

441 - 450 146 

460 - 469 14B- 

4B0 - 490 150 

502 - 512 152 

524 - 535 154 

549 - 560 156 

575 - 5B7 15B 

602 - 616 160 

632 - 647 162 

664 - 6B0 164 

698 - 716 166 

736 “ 754 16B 

776 - 796 170 

820 - 842 172 

868 - 893 174 

921 - 948 176 

979 - 1009 178 

1043 - 1077 IBO 

1115“ 1153 182 

1 195 “ 1238 184 

1286 - 1335 186 

1389 “ 1445 188 

1508 - 1573 190 

1645 “ 1721 192 

1806 - 1896 194 

199B - 2107 196 

2229 - 2363 198 

2515 “ 2684 200 

2877 - 3095 202 

3349 - 3643 204 

3991 - 4407 206 

4916 “ 5549 208 

6362 “ 7439 210 

8941 - 1 1 173 212 

14B40 - ,22020 214 

42232 - 466033 216 



07-07-1986 



! monthly population: 




Popul at i on 


range - Sample 


Size 


291 




296 


125 


303 


- 


308 


127 


314 


- 


320 


129 


327 


- 


332 


131 


340 


- 


345 


133 


353 


- 


359 


135 


367 


- 


374 


137 


382 




389 


139 


398 


- 


405 


141 


415 


- 


422 


143 


432 


- 


440 


145 


451 


- 


459 


147 


470 


- 


479 


149 


491 


- 


501 


151 


513 


- 


523 


153 


536 


- 


548 


155 


561 


- 


574 


157 


588 


- 


601 


159 


617 


- 


631 


161 


640 


- 


663 


163 


681 


- 


697 


165 


717 


- 


735 


167 


755 


- 


775 


169 


797 


- 


819 


171 


843 


- 


867 


173 


894 


- 


920 


175 


949 


- 


978 


177 


1010 


- 


1042 


179 


1078 


- 


1114 


181 


1154 


- 


1194 


183 


1239 


- 


1285 


185 


1336 


- 


1388 


187 


1446 


- 


1507 


189 


1574 


- 


1644 


191 


1722 


- 


1805 


193 


1897 


- 


1997 


195 


2108 


- 


2228 


197 


2364 


- 


2514 


199 


2685 


- 


2876 


201 


3096 




3348 


203 


3644 


- 


3990 


205 


4408 


- 


4915 


207 


5550 


- 


6361 


209 


7440 


- 


8940 


21 1 ' 


1 1174 


- 


14047 


213 


22021 


- 


42231 


215 


465914 


and 


above use • • • • 


217 
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RSED V3.0 



Table o-f Adjustment Factors 



FOR DEFECT RATE ADJUSTMENT 



OVER '/. - THRU 7. FACTOR 7. 

UP to’ - 1.0 0.46 

2.0 - 3.0 0.78 

4.0 - 5.0 1.00 

6.0 - 7.0 1.17 

8.0 - 9.0 1.31 

10.0 - 11.0 1.44 

12.0 - 13.0 1.54 

14.0 - 15.0 1.64 

16.0 - 17.0 1.72 

IB.O - 19.0 1.80 

20.0 - 21.0 1.87 

22.0 - '23.0 1.93 

24.0 - 25.0 1.99 

26.0 - 27.0 2.04 

28.0 - 29.0 2.08 

30.0 - 31.0 2.12 

32.0 - 33.0 2. 16 

34.0 - 35.0 2.19 

36.0 - 37.0 2.22 

38.0 - 39.0 2.24 

40.0 - 41.0 2.26 

42.0 - 43.0 2.27 

44.0 - 45.0 2.28 

46.0 - 47.0 2.29 

48.0 - 49.0 2.29 



07-07-1986 

FOR DEFECT RATE ADJUSTMENT 



OVER 7. - THRU' 7. FACTOR 7. 

1.0 - 2.0 0.64 

3.0 - 4.0 0.90 

5.0 - 6.0 1.09 

7.0 - 8.0 1.24 

9.0 - 10.0 1.38 

11.0 - 12.0 1.49 

13.0 - 14.0 1.59 

15.0 - 16.0 1.68 

17.0 - 18.0 1.76 

19.0 - 20.0 1.84 

21.0 - 22.0 1.90 

23.0 - 24.0 1.96 

25.0 - 26.0 2.01 

27.0 - 28.0 2.06 

29.0 - 30.0 2.10 

31.0 - 32.0 2.14 

33.0 - 34.0 2. 17 

35.0 - 36.0 2.20 

37.0 - 38.0 2.23 

39.0 - 40.0 2.25 

41.0 - ' 42.0 2.26 

43.0 - 44.0 2.28 

45.0 - 46.0 2.29 

47.0 - 48.0 2.29 

49.0 - 50.0 2.29 
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KHERE SOME DEFICIENCIES ARE NOT REWORKED WITHIN ALLOTIED TIME 



a. Price for specified services 

b. Population of services this period being billed 

c. Price per service 

d. Number of services sampled 

e. Number of sampled services rejected 

f. Observed defect (reject) rate ((e/d)xl00j 

g. Adjustment factor Note 1 

h. Deductible defect rate (f - g) Note 2 

i. Extrapolated defects [Whole number portion of (h x b)/100] 

j. Defects observed outside the sample Note 3 

k. Acceptable rework completed 

l. Net number of services to deduct (i - k) Note 2 

m. Deduct for unsatisfactory services (c x 1) 

n. Liquidated damages (lOi of c x k) Note 4 
0 . Other adjustments (- for deduct) Note 5 

p. Total Payment (a - m - n ^ o) 

Notes-: 1. Adjustment factor varies with observed defect rate. 



$5,000.00 
1,000 
$ 5.00 

100 
5 

5.00t 

1 . 00 % 

4.00% 

40 

23 

3 

37 

$ 185.00 

i 1.50 

$ 0.00 

$4,813.50 



2. Cannot be less than zero (0) 

3. Defects observed outside the sample will not be used as 
a basis for extrapolation but shall be considered in 
calculation of payment for satisfactory rework. 

4. Liquidated damages calculated as appropriate in accordance 
with contract provisions. 



5. This could include a deduction for work performed by Government 
or other forces or payment for rework from a prior billing period 
provided it would not have resulted in less than zero Net 
Services to deduct (item 1 above) in that period. 
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